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FOREWORD 


Understanding the drivers of vaccine confidence and, ultimately, increasing 
uptake, have been a focus of scientists, governments, and healthcare 
providers alike, dating all the way back to the first smallpox vaccines. 
Social media has become a valuable tool for the global health community 
to listen, understand, and more uniquely address the drivers of vaccine 
confidence at a scale not previously seen. If properly leveraged, we believe 
that online platforms can play an increasingly meaningful role in improving 
access to information, both for COVID-19 and beyond. Realizing this 
opportunity will require a whole-of-society approach, both by platforms and 
by other health and media stakeholders. It also requires us to effectively 
learn from what is working, so that we can replicate and scale these 
approaches, as well as learn what is not working, so that we can further 
innovate and improve. 


Heidi Larson, PhD, Head of the 
This new pathway to building trust and increasing vaccine confidence is i oe 
the focus of Advancing Health Online (AHO). As the first initiative of the and Tropical Medicine 

AHO, the Vaccine Confidence Fund (VCF) and its report seeks to share 

insights from the 33 grantees, to rapidly increase knowledge in the global 

health community and beyond. We feel it is important to share the findings from this report widely so 


that marginalized and underserved populations, and the world at large, can benefit from this research. 


VCF is playing a crucial role in leveraging 
FUNDERS NOTE innovative research to demonstrate new 


by Kang-Xing Jin, Head of Health, Meta Platforms Inc. ways in which social media platforms can 


and Michael D. Fornwall, Associate Vice President, play a positive role in societal health. It is 
Vaccination Confidence, Merck & Co. Inc., Rahway NJ, about listening and understanding, engaging 


ee DE outeidethe United States and and educating, and then measuring impact. 
This new medium presents a multitude of 

When Meta and MSD announced Advancing opportunities, as well as challenges. As we 

Health Online (AHO) last June, we were still have much to learn, it also demands new 

excited to come together with key global science and novel approaches. Overall, VCF 

health partners and stakeholders around the seeks to expand our collective knowledge of 

world with a mutual goal of leveraging social the role that social media can play in increas- 

media to advance health outcomes. Today, ing vaccine confidence, and it comes not a 

we are delighted to see results from some of minute too soon. 

the Vaccine Confidence Fund's grantees. We 

are excited to learn from these results in our 

continued effort to better understand how 

social media and digital platforms can be used The Vaccine Confidence Fund is managed by Global 


to positively impact people and communities Impact and financially supported by Meta and Merck & 
Co. Inc., Rahway NJ, USA (known as MSD outside the 

around the world. United States and Canada) (“MSD”). Meta and MSD did 
not participate in the selection of grant recipients. 
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EXECUTIVE SUMMARY 


It has been more than two years since the World Health Organization 
(WHO) officially declared COVID-19 as a global pandemic. Two years 

in, we have more information about COVID-19, its effects, and how to 
combat it (especially through highly effective vaccines), but the world is 
still at risk as new and more transmissible variants continue to emerge 
and the prevalence of conflicting information and uncertainty about the 
future continues to pervade our societal consciousness. Omicron took 
the world by surprise in November 2021 and now its BA.2 subvariant is 
rapidly spreading around the globe. 


The pandemic has had a huge impact on global health—and not just 
because of the direct effects of COVID-19. People have now experienced 
two years of lessened physical activity due to extended lockdowns 

and larger numbers working from home. People have missed annual 
check-ups with their general physicians or ignored worrisome symptoms 
because of limited access to health services or the fear of contracting 
COVID-19 at the doctor's office. The world’s overall health, both physical 
and mental, has suffered tremendously because of the pandemic and its 
many ripple effects. 


“Social 
media is a 
powerful, 
constantly 
evolving 
tool that 
is shaping 
opinions 
and 
behaviors 
across the 
globe.” 


- Heidi Larson, PhD, Head 
of the Vaccine Confidence 
Project at the London 
School of Hygiene and 
Tropical Medicine 


The biggest tool in our toolbox to fight COVID-19 and regain some sem- 
blances of normalcy are the many vaccines that have been developed and 
distributed around the world (albeit in a widely inequitable manner). Large 
populations still lack access to those vaccines and there are hundreds 

of millions of people who doubt the science and refuse to get vaccinated 
due to the prevalence of conflicting information. 


There are many issues currently impacting vaccine confidence, including 
continued debates on infection-induced versus vaccine-induced immu- 
nity; the public perception that COVID-19 is “over” and people are ready 
to start living normally; and a myth about “Vaccine Acquired Immune 
Deficiency Syndrome”. 


One way to address issues associated with vaccine confidence and 
uptake is the use of social media. 


“Social media is a powerful, constantly evolving tool that is shaping 
opinions and behaviors across the globe,’ said Heidi Larson, PhD, Head 
of the Vaccine Confidence Project at the London School of Hygiene and 
Tropical Medicine, and Advancing Health Online (AHO) collaborator. 


Just under half of the world population, or 3.47 billion people, use social 
networks". Millennials and Gen Zers are “the most active online... aver- 
aging five digital platforms daily.” Because the global population spends 
so much time online, it's only natural that people seek out health infor- 
mation, and COVID-19 information, in the digital spaces they frequent. In 
fact, “nearly 90% of all adults in the USA search for health information on 
Facebook, Twitter, YouTube, and other social media sites.” 


With the above in mind, Advancing Health Online (AHO) formed as an 
initiative to advance public understanding of how social media and 
behavioral sciences, can be leveraged to improve the health of commu- 
nities around the world. 


AHO launched the Vaccine Confidence Fund? (VCF) its inaugural 
initiative, in June 2021. VCF's goal is to support targeted research that 
addresses the primary research question of how best to utilize social 
media and online platforms to better understand and increase vaccina- 
tion confidence and uptake both in the context of the current pandemic 
and beyond. 


VCF is focused on supporting research that increases access and 
equity regarding vaccination confidence and uptake, particularly among 
historically marginalized or excluded communities globally. 


The Fund's key principles (see Appendix 3) are equity, community 
engagement, interdisciplinary collaboration, external validity, actionable 
focus, and transparency and information sharing. 


VCF sought applicants from interdisciplinary teams and cross-sectoral 
partnerships to develop truly transformative proposals that would 
address critical research gaps with creative solutions. Thirty-three 
teams received grants averaging around US$220,000 for a total of over 
USS$7 million disbursed. Advancing Health Online is excited to share the 
VCF's report that provides an overview of the 33 grantees’ projects, key 
findings, and more detailed results. 
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AHO’s Mission 


AHO’s mission is to 
bring together actors 
from the technology, 
health, global devel- 
opment, and 
academic sectors to 
advance understand- 
ing of how social 
media and online 
engagement can best 
be utilized to better 
understand and 
increase health and 
resiliency of commu- 
nities around the 
world. Given the 
urgency COVID-19 
created, the first area 
of focus is on vaccine 
confidence, but lon- 
ger-term, the AHO will 
be looking across 
health topics more 
broadly with a stated 
goal of driving posi- 
tive health action 


offline. 


1 “44.6% of the total world population uses social networks.” eMarketer, November 2021. Social Network Users Worldwide, 2021-2025 Forecast 


2. Meta and Merck & Co., Inc., Rahway NJ, USA (known as MSD outside the United States and Canada) are financially sponsoring the Fund, which 


is independently managed by Global Impact. 
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BACKGROUND 


The past two years have brought an unprecedent- 
ed shift in the way we see the world and engage 
with others. The global COVID-19 pandemic has 
caused a reduction in face-to-face communication 
and an increased reliance on social media for 
sharing memes, news, and information about 
health and politics. Nowhere have we seen this 
more than in the conversation around COVID-19 
and the vaccines that have been produced to 
combat its spread’. The pandemic has had a huge 
impact on global health—and not just because 

of the direct effects of COVID-19. People have 
experienced more than two years of lessened 
physical activity due to extended lockdowns 

and larger numbers or people working from home. 
People have missed routine immunizations and 
annual check-ups with their general physicians or 
ignored worrisome symptoms because of limited 
access to health services or the fear of contract- 
ing COVID-19 at the doctor's office. The world’s 
overall health has suffered tremendously because 
of the pandemic and its many ripple effects. 


The biggest tool in our toolbox to fight COVID-19 
and regain some semblances of normalcy are the 
many vaccines that have been developed and 


distributed around the world (albeit in an inequita- 
ble manner). Large populations still lack access to 
those vaccines and there are hundreds of millions 
of people who have doubts or anxieties and 
refuse to get vaccinated due to the prevalence of 
conflicting information’. So, what role does social 
media play? And, what insights can we derive 
from a more detailed exploration into data points 
and trends that can help individuals, researchers, 
healthcare practitioners, and governments make 
better choices about their own personal health, 
support decision making for others and better 
understand concerns? 


VCF is the first initiative of Advancing Health 
Online (AHO). Guided by the principles of equity, 
interdisciplinary collaboration, actionable focus, 
community engagement, external validity, and 
transparency and information sharing, it sought to 
explore the primary research question: 


“How can social media? be best utilized to 
understand and increase vaccination confidence 
and uptake?” 


3 Molla, Rani. “How Coronavirus Took Over Social Media: The Pandemic Quickly Became One of the Most Talked-About Things on the 
Internet” Recode, 12, March 2020", https://www.vox.com/recode/2020/3/12/21175570/coronavirus-covid-19-social-media-twitter-face- 


book-google. “Sprinklr counted a record nearly 20 million mentions of coronavirus-related terms on March 11 [2020]. For context on just 
how big that number is, mentions of the newly canceled NBA games were under 2 million and mentions of Trump were about 4 million on 


that day.” 


4 Hannah Ritchie, Edouard Mathieu, Lucas Rodés-Guirao, Cameron Appel, Charlie Giattino, Esteban Ortiz-Ospina, Joe Hasell, Bobb 


e 


Macdonald, Diana Beltekian and Max Roser (2020)-“Coronavirus Pandemic (COVID-19)”. Published online at OurWorldInData.org. 
Retrieved from: ‘https://ourworldindata.org/coronavirus’ “As of 1:46 PM on 4/14/22, 11.4 billion doses of COVID vaccines have been given, 


4.6 billion people are “fully vaccinated,” and 59% of the world is “fully vaccinated.” "64.8% of the world population has received at least one 
dose of a COVID-19 vaccine. 11.42 billion doses have been administered globally, and 12.23 million are now administered each day. Only 
4.8% of people in low-income countries have received at least one dose." 


5 The Fund's definition of social media is meant to be broad and inclusive. Our social media definition includes: traditional social media, 
(e.g., Facebook, Instagram, Twitter, LinkedIn, YouTube, TikTok, SnapChat); search engines (e.g., Google, Bing, etc.) and other online 
platforms that gather user data and micro-target users; and online chatbots. This Fund recognizes that social network data analyses and 
other big data analytical techniques may also be leveraged as analytical methods in prospective research proposals. 
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VACCINE CONFIDENCE FUND OVERVIEW 


-A lj- EQUITY 
N i A ne a 
wv v Focus on historically marginalized or 


excluded populations globally given the 
significant barriers those populations 
experience 


ao INTERDISCIPLINARY 
COLLABORATION 


as Encourages interdisciplinary 
collaboration and cross-fertilization of 
ideas from non-traditional disciplines 


( d 


Immediately actionable and valuable 
for the global health community 


fo ACTIONABLE FOCUS 
a al 


With nearly 300 applications, the interest in 
exploring this new and growing area of research 
was Clear. More and more communities are 
relying on social media as a significant source 
of information about COVID-19 vaccines. With 
the scale of these digital and online platforms, 

it is vital that we improve our understanding of 
the role these platforms can play and effectively 
harness their capabilities to increase vaccination 
confidence and contribute to vaccine decision- 
making. The VCF is focused on supporting 
research to effectively increase vaccination 
confidence, particularly among historically 
marginalized or excluded communities globally. 


e29 COMMUNITY ENGAGEMENT 


ER Proactively and authentically partner 
with communities they are working on 
throughout project/research life cycle 


7@~, EXTERNAL VALIDITY 


di o Surface insights that are more 
oY universally valid and potentially 
scalable 
Q J g TRANSPARENCY AND 
(i)o INFORMATION SHARING 
€ È » Rapidly disseminating insights, data, and 


learning as public goods to contribute to 
ongoing or future vaccination campaigns 
and to advance the global agenda 


A. CAPTURING COMMUNITY VOICES 


A Mayan health researcher interviews a study 
participant for the UCSF project exploring trusted 
messengers and social networks in indigenous 
communities in Guatemala. 
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VCF RESULTS 


VCF DEVELOPMENT AND SELECTION 


To identify a robust and diverse cohort of grantees to receive support through the VCF, the fund 
launched an open application process in the summer of 2021. The VCF REP was open for eight weeks 
from June 10 to August 6, 2021, and in that time, received 281 applications from 48 countries across six 
continents. 


Each of the 281 applications received was reviewed in a preliminary screening by the Fund Manager. 
The top third advanced to one of thirteen Advisory Council members based on their respective expertise 
for a second round of review, in addition to further review by the Fund Manager. 


Finally, of those applications, 33 grantees (11.7% of total applications) were chosen based on the 
defined criteria by an independent panel of expert reviews from the VCF Advisory Council and the Fund 
Manager. 


Each application was scored against twelve criteria and received a score of 1, 3, or 5 in each category 
with a maximum total score of 60. Criteria included: 


e Importance e Feasibility e Collaborative partnerships 
e Alignment e Novelty e Applicability 

e Equity e Engagement e Impact 

e Experience e Urgency e Budget 


GRANTEE SNAPSHOT 


AMOUNT AWARDED 
281 grant 
applications 
received 
Over 


shel $7.3 million 


disbursed to the 
33 grantees 


NUMBER OF GRANTEES 


GEOGRAPHICAL DISTRIBUTION 


Number of grantees in each country 


N 


_ 


SPECIAL POPULATION 
FOCUS 
MEASUREMENT 
Confidence or —— 
other (48%) 
Closed loop 
feedback (52%) 


[01 | Black 


AGE 

ican 
SELLLLLLL 
AELLLLLLL 
AERTATTTTT 
ALLL. 
Perera 
AELLLLLLL 
ey 
Perera 


o 0 e 


yt 


(ya Indigenous 


03 


05 


+> Women 


© Persons with Disabilities 


71% 

of grantees 
focused on 
adults 


26% 

of grantees 
focused on 
youth 


3% of 
grantees 
focused on 
the elderly 


Health Workers 


Pregnant and 
post-partum women 


ECONOMY 


Latinx 


High 


income Both 


LGBTIQ+ 


Other 


TECH PLATFORMS 


Facebook 
Instagram 
YouTube 
Twitter 
TikTok 


WhatsApp/ 
messaging 


ChatBots 
Influencers 
Crowdsourcing 


Other/multiple 


Low & 

middle 

income 
(18) 
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CONTENT 


APPROACH 


MEASUREMENT 


BUDGET 


Importance: Does the proposal contain a coherent and compelling problem statement? 


Alignment: Do the proposal’s research question(s) and research methods align with the Fund’s primary research 
question? 


Equity: Does the proposal aim to support focus communities, or address underlying drivers of disparities, especial- 
ly in trying to achieve health equity? 


Experience: Does the proposed research team have a proven track record of expertise in the proposed fields of 
study? 


Feasibility: Are the proposed research methods technically and methodologically feasible given time and cost 
constraints (see more details below)? Do the proposed research methods adequately and appropriately consider 
privacy — specifically do the methods design for user consent? 


Novelty: Is the proposed line of inquiry novel and does it have the potential to fill an important knowledge gap? 


Engagement: Does the proposed research incorporate engagement with the focus community before, during, and 
after the research? 


Urgency: Does the approach reflect the immediate circumstances of the targeted community? Are there any risks 
that those circumstances may change in the immediate future? 


Collaborative partnerships: Does the proposed research team reflect interdisciplinary areas of expertise? Does 
the proposed research team pursue partnerships across sectors/disciplines to generate bold and innovative 
solutions? 


Applicability: Can the anticipated research results be used to shape and inform practical public health and 
technology interventions in the short-term? How can the results of this research be applied in communities beyond 
those studied? 


Impact: Does this proposal articulate clear measures and their pathway to immediate or potential impact (i.e., 
vaccine uptake)? If impact data is not available (due to the project timeline), are there interim results that can be 
tracked? 


Budget: Does the proposal include a budget and clear justification for key budget items? 


Á SELECTION CRITERIA 


Each application was scored against 12 criteria and received a score of 1, 3, or 5 in each category with a 
maximum total score of 60. 


Grantees were informed of their selection in September 2021 and began work on their research projects shortly 
thereafter. Mid-term reports were submitted in December 2021 and presented to the Advisory Council, and final 
reports (for the majority of grantees) were submitted in March and April 2022. The limited window of time for 
grantees to implement their projects (less than six months) was by design, with the intention to get information 
out to the broader community as quickly as possible and to continue to iterate and build on the work. This report 
represents findings from all 33 grantees. 


KEY FINDINGS 


The sections below highlight common key findings shared across multiple studies. Further insights and 
findings can be found in the detailed findings section. 


Customizing messaging: Messages tailored 
to the intended audience have a higher chance 
of resonating. Larger mass information cam- 
paigns that focus on general information may 
be less effective than engagement activities 
and messages tailored for specific geogra- 
phies and communities (e.g., language and 
cultural context). This finding has the potential 
to impact how government and healthcare 
providers fund and share information intended 
to increase vaccine confidence and decrease 
vaccine hesitancy. 


WY SCOPE OF THE VCF IMPACT REPORT 


Total numbers of people reached (viewed or 
interacted with social media), engaged (participated 
in survey activity, etc.) and vaccinated as reported by 
grantees 


Successfully reaching populations: It is an 
exciting new way to engage many people for 
the betterment of their health. Social media 
presents a key opportunity to share tailored 
information, provide a neutral platform to 
discuss and engage around vaccines and 
vaccinations, and tap into the social power 
of influencers and trusted members of the 
community. As noted, these factors are key to 
impacting perceptions and behaviors around 
vaccines. At the same time, recruiting study 
participants and reaching the specific popu- 
lations can sometimes be tricky. Continuing 
to expand our understanding of how best to 
harness social media for global health will be 
key when delivering appropriate messages to 
marginalized and underserved communities. 


34,273,993 


people reached 


106,030 


people vaccinated 
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Providing safe spaces: Many of the demographics 
studied, especially those who are vaccine hesitant, or in 
a position of responsibility (i.e., parents and healthcare 
workers), are more likely to engage in online conver- 
sations in small, trusted groups. Many of the studies 

in this report noted that either providing a “safe” online 
space to discuss concerns among the vaccine hesitant, 
gaining access to trusted physicians or medical profes- 
sional who could answers specific questions, or being 
able to engage on platforms with other individuals 
from similar demographic backgrounds (i.e., mothers) 
often had significant impact in increasing vaccine 
confidence. 


Engaging trusted voices: A key finding was the 
importance of trusted community members. Hearing 
representations of themselves in communications was 
key for many communities. For many of the commu- 
nities participating in the studies, there was mistrust 

of the government and health organizations. However, 
what the research does highlight is that while there was 
not necessarily consistency in who those trusted sourc- 
es were, identifying them and having them play a role in 
sharing information and engaging the community was 
likely to have a positive impact on vaccine confidence. 
Finally, the influence of families, as well as concern for 
their health and wellbeing were common drivers both 
for and against vaccination. 


“In the context of 
polarization and 
vaccine mandates, 
discussing COVID- 
vaccination has 
become problematic 
among healthcare 
workers in Belgium. 
Creating safe spaces 
for dialogue among 
healthcare workers 

is a multi-faceted 
effort that requires 
adapting to the 
changing pandemic 
context, offering 
multiple modes of 
communication, 
having active dialogue 
moderation, preventing 
an echo chamber of 
misinformation, and 
putting aside the aim of 
increasing vaccination 
uptake to promote 
trust.” 


- ITM Antwerp 


DETAILED FINDINGS 


This section provides a more detailed look into some of the insights coming out of the 33 research 
projects conducted under the Vaccine Confidence Fund (VCF). It explores findings by types of 
vaccinations, key population groups, and by methodology type. Further information on the studies 
mentioned can be found in the Appendix and links in the footnotes. 


VACCINATIONS 


VACCINE UPTAKE 


Insights on what drives vaccine uptake is one of the key 
areas the VCF, and the global health community, are eager 
to share and understand. However, obtaining data can be 
complex and restricted in many countries, even with self-re- 
ported approaches. 


e |n one study, conducted by the International Longevity 
Centre, click through rates for vaccine appointments 
were used as a proxy to measure uptake following 
engagement with their social media campaign. The 
study found that “(s)ocial media campaigns targeting 
older adults in deprived areas directly have significant 
potential to increase vaccine uptake — especially for less 
well-known vaccines.” This is potentially a cost-effective 
way to improve vaccination rates, especially for older 
generations in underserved communities. 


e Another study, by Mali Health’, that looked at vaccine 
uptake noted great success in increasing confidence and 
trust in the COVID vaccine, but they did run into some 
issues with access and availability of appointments. 
While 252 of 354 users pledged to get the vaccine only 
18 of 354 received a vaccine during the period of study. 
It is quite likely that this number would have been higher 
with a longer timeline® 


6 International Longevity Centre — UK, “Leveraging intergenerational relations to 


“lam not against vaccination 
in general, but in relation to 
vaccination against Covid-19, | 
received a lot of contradictory 
information from my friends 
or through social networks; 
which created doubts and 
uncertainties that led to 
hesitation and even fear of the 
Covid-19 vaccine on my part. 
My fear started to dissipate 
when | installed Kenéya Blon 
in my phone and received the 
correct information. | then 
made the decision to get 
vaccinated after discussions 
that | had the opportunity to 
have with the health staff 
through the application. | 
have already gone to the 
health center to register for 
the vaccination when it is 
available. | am just waiting 
for the health center to call 
me for an appointment to get 
vaccinated.” 


- Comments from Mme. Bathily, shared with 
Mali Health (22 February 2022) 


increase uptake of immunization in older Black communities.” The cost per view of other healthcare adverts (2021): 1-15 cents compared 
to 0.5 cents (ILC’s campaign). In addition, test campaigns generated 1,179 unique booking clicks, across all audience ages and vaccina- 


tion types. 


7 Mali Health, “Use of voice-based social media messages to improve vaccination knowledge and confidence in underserved peri-urban 


communities.” 
8 The study ran from 23 September 2021 — 28 February 2022. 
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ROUTINE VACCINATION 
Several projects funded by the VCF looked at the role of social media in vaccinations beyond COVID-19. 


One of these studies, conducted by ORB International, examined perspectives 
of mothers and daughters on the HPV vaccination in Brazil. The study found 
that lack of awareness, both for the vaccine and the potential health impact of 
HPV, was the largest barrier to vaccination. Because there are several doses 
that need to be received within a certain time frame, the process and availabili- 
ty of the vaccine was another issue the study noted. The study also found that 
cancer prevention was one of the most motivating messages to mothers. As 
with several studies, seeing themselves reflected in communications was also 
noted as important to the mother and daughter participants. 


Another study by the International Longevity Centre found, somewhat sur- 
prisingly, that “(d)espite popular stereotypes to the contrary, targeting older 
users directly on social media is a more effective way of encouraging them 
to get their jabs than engaging them indirectly through younger people.”'® 
The study focused on older generations and looked at the pneumococcal 
vaccine in underserved and marginalized communities (particularly focusing 
on individuals of African and Black Caribbean backgrounds). It also found 
that social media was a more cost-effective way of encouraging margin- 
alized older adults to get vaccinated. “Cost per view of other healthcare 
adverts (2021): 1-15 cents compared to 0.5 cents (our campaign)’."’ Key 
recommendations directed to the NHS focused on investing more funding 
in targeted social media campaigns to improve uptake in those most at risk 
and least likely to get vaccinated. 


Rising concerns about the COVID-19 vaccine are also impacting parental 


confidence in all vaccines according to interviews, online discussions and fo- 


cus groups, conducted as part of a study conducted by Unity Consortium," 
which looked at how to better understand the elements necessary to engage 
and educate parents through social media about adolescent and young 
adult vaccination. The study also noted that the conflicting information has 
increased stress and uncertainty, and it increased the amount of research 
parents are doing to feel confident in their decision making. 


“Despite 
popular 
stereotypes to 
the contrary, 
targeting 

older users 
directly on 
social media 

is amore 
effective way 
of encouraging 
them to get 
their jabs 

than engaging 
them indirectly 
through 
younger 
people.” 


ORB International, with Twist MKTG Ltd, dbs Real Chemistry, Fiocruz/NIESP VCP, “A Spread on Two Fronts: Informing HPV Inoculations in 


the Time of COVID-19 Misinformation.” 


International Longevity Centre — UK, “Leveraging intergenerational relations to increase uptake of immunization in older Black 


communities.” 


International Longevity Centre — UK, “Leveraging intergenerational relations to increase uptake of immunization in older Black 


communities.” 


Unity Consortium, “Activating Vaccine Supportive Parents of Adolescents and Young Adults as Trusted Voices on Social Media” 


COVID-19 VACCINATION 
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Grantees who focused their research on COVID-19 vaccination specifically identified several key themes. 


Importance of a tailored approach: The studies 
conducted under the VCF grant showed that there are 
different reasons for vaccine hesitancy across different 
communities, but addressing specific concerns and 
questions was key. This diversity of concern and need 
speaks to the necessity for government and healthcare 
providers to share information and provide resources in 
amore tailored approach. 


Concerns around vaccine safety and lack of trust: 
Throughout many of these groups, concerns about 
vaccine safety were a major issue for many, and the 
issue of trust in healthcare providers or governments 
was another commonly seen finding. 


Specific messaging found to be most effective: 
Several studies also showed that the way in which 
information is communicated can be key in persuading 
those who may be hesitant. 


e A study by the University of Washington 
(Department of Human Centered Design & 
Engineering), showed that “communicating the 
risks of COVID-19 vaccine side effects using verbal 
probability descriptors (e.g., common, rare) was 
associated with higher, less accurate, and more 
variable perceptions of risk than using numerical 
probability descriptors (e.g., 4%, 0.0035%)." The 
same study also was able to show that sharing the 
amount and consistency of scientific research on 
COVID-19 vaccines increased awareness and, with 
a transparent approach, people were more likely to 
trust this information than the general information 
available on government websites. 


Data For Society 


Have any thoughts or concerns about the 
COVID-19 vaccine? Tell us! 


There have been a lot of 
rumors circulating about side 
effects of the COVID-19 
vaccine. Here are several 
*unproven rumors* about the 
COVID-19 vaccine. 


No, covid-19 vaccines do NOT 
make you magnetic. 


It is normal to experience 
mild side effects like a 
headache, chills, fever, 
nausea and pain at the 
point of injé on ofter 


getting v i iGted. 


A. UNPROVEN RUMORS 


University of Chicago, Testing interventions to 
address vaccine hesitancy on Facebook in East 
and West Africa 


e Infact, several VCF studies'* have showed that more specific information on the medical impact of 
COVID-19 was more effective than general or non-specific statements. We also saw that “although 
we often think more concise messages or research summaries may be better, our findings suggest 
that providing more transparent and complete information about each study—including studies with 


divergent findings—can lead to a more trustworthy, informative, and compelling message.” 


13 University of Washington (Department of Human Centered Design & Engineeri 


Trust of COVID-19 Vaccine Research on Social Media.” 


14 The George Washington University, “Empathic Engagement with the Vaccine 


15 The George Washington University, “Empathic Engagement with the Vaccine 


ng), “Large-scale Stu dies to Advance Public’s Access and 


esitant in Online Spaces.” 


esitant in Online Spaces.” 
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KEY POPULATIONS 


HEALTH CARE WORKERS 


Several grantees focused their research specifically on health care workers (HCW) and how their 
perspectives on vaccine confidence influenced hesitancy and uptake in the communities where they 
worked. As trusted sources of information for vaccines, understanding hesitancy in this community is 
especially important so that HCW can receive the information and space they need to feel comfortable 
with vaccination, and share that information with their patients. Healthcare workers often play a key role 
as trusted sources of information in the community and are often sought as the first source of advice. 
They can also provide information on how and where to obtain vaccines and address any specific issues 
patients may have. Therefore, increasing vaccine confidence amongst HCW potentially has exponential 
impact on increasing vaccine confidence and uptake. 


The below excerpt from an article external to the VCF, “The Vaccine-Hesitant Moment”, published in the 
New England Journal of Medicine, highlights the key trust that people have in health care workers on a 
global scale and also their potential impact: 


“Physicians and other health care providers are still among the most trusted persons when it 
comes to health care advice. The Wellcome Global Monitor surveyed people in 140 countries 
and found that 73% of the respondents said that they would trust a doctor or a nurse more 

than others; the percentage was 90% in the higher-income countries. Vaccine acceptance can 
increase, but health care providers need to offer support and encouragement and listen to what 
matters from the patients perspective. Equipping physicians with information on the nature and 
scope of circulating concerns in their communities may help them address such concerns in 
the clinic, while also informing appropriate interventions at the community level.”"° 


e A study looking at the vaccine confidence of healthcare workers in Belgium, conducted by the 
Institute of Tropical Medicine, Antwerp,'” sought to explore the role of safe spaces for dialogue. HCW 
occupy a particular space as they face COVID-19 care and prevention due to efforts to vaccinate 
more directly through their employment. HCW often experience difficulties talking about COVID-19 
candidly, especially those who may be vaccine hesitant. Exploring the scale of HCW from very 
confident to more vaccine hesitant, researchers found frustrations, mistrust, and preconceptions 
amongst HCW from these two different camps. “Vaccine confident participants felt unvaccinated 
colleagues were quite defensive or that it was a lost cause trying to convince them. On the contrary, 
vaccine skeptical participants felt they were often considered egocentric, and their concerns not 
taken seriously.” The study found that while confidence was important, providing a safe space online 
for virtual face-to-face dialogue helped to relieve frustrations (especially for vaccine hesitant HCW) 
and was preferred to text-based engagement. The study also found that “(c)reating safe spaces for 
dialogue among healthcare workers is a multi-faceted effort that requires adapting to the changing 


16 Heidi J. Larson, Ph.D., Emmanuela Gakidou, Ph.D., and Christopher J.L. Murray, M.D., Ph.D.. (2022). The Vaccine-Hesitant Moment. The 
New England Journal of Medicine. 387;1, 58-65. https://www.nejm.org/doi/pdf/10.1056/NEJMra2106441 


17 Institute of Tropical Medicine Antwerp, “Cultivating Online Safe Spaces: Addressing unspoken hesitancy to build vaccine confidence in 
healthcare workers in Belgium.” 
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A BUILDING TRUST 


Local participants learn about the 
Sabalibougou app at a promotional event held 
by Mali Health 22/2/2022 in the community of 
Sabalibougou in Bamako, Mali. 
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pandemic context, offering multiple modes of communication, having active dialogue moderation, 
preventing an echo chamber of misinformation, and putting aside the aim of increasing vaccination 
uptake to promote trust.” The study calls for leveraging online media to foster greater “vaccine 
dialogue, trust in democratic science-making and vaccine confidence by listening to vaccine narra- 
tives and addressing vaccine concerns and information deficits and creating venues for dialogue 
and peer support.” 


VACCINE CONFIDENCE AMONGST HISTORICALLY EXCLUDED AND MARGINALIZED 
COMMUNITIES 


Members of historically excluded and marginalized communities, who have been denied access to 
systems in the past, faced that same exclusion in the context of the pandemic. One of the unsurprising 
key findings is that understanding the community, its history, and current concerns is key in increasing 
uptake and confidence, as well as taking language into account. Furthermore, engaging trusted com- 
munity figures and influencers identified by the community proved to be useful in increasing vaccine 
confidence. 


Looking at minority populations in 4 countries'®, the Minority Rights Group found that there were 
significant and deep rooted differences between minority and majority populations and “thus 
treating diverse populations as homogenous in a misguided attempt to save time or money or 

to promote efficiency entails a high risk of back-firing and can mean efforts do not reach their 
intended targets."?? Working closely with trusted local minority organizations, researchers found that 
when data is “disaggregated by ethnicity, language and religion, findings help explain issues such 

as residual pockets of non-vaccination in Sri Lanka, or the fivefold difference in vaccination rates 
between different areas of Kenya.” The research emphasizes the ethnic, religious, and linguistic 
barriers to vaccine access and uptake. This likely means more locally relevant information, in local 
languages and through trusted community figures, will have a greater impact than general, centrally 
broadcasted information. The study also showed that while social media findings can be very useful 
for areas with high levels of usage, for those in more remote areas, social media may not have the 
reach needed to impact the community, both in terms of actual signal and in terms of a presence of 
trusted community figures engaging online. 


Another study, conducted by Oklahoma State University, looked at the disproportionate impact of 
COVID-19 among marginalized Hispanic immigrant populations in the US, as well as the lack of 
research on vaccine confidence among these populations. The study found that “sourcing culturally 
specific narratives enhance message persuasiveness; here confidence in the COVID-19 vaccine 
among Hispanic survey participants is increased when exposed to culturally specific expressions 

of vaccine confidence derived from Hispanic communities.’”° The study also highlighted the impor- 
tance of amplifying “organic community narratives with key mutually understood cultural nuances.” 
It also found that in addition to a preference for images that looked like themselves, themes focused 
on Hispanic cultural themes of collectivism and community were more preferred. 


See Minority Rights Group, with Grand Synergy Development Initiative (GSDI), Verite Research Pvt Limited, Bytes for All, “Diversity: Impact 
on Vaccine Equality (DIVE).” Algeria - focus on Amazigh/Tamazight speakers, Kenya - focus on Somali population, Pakistan - focus on 
religious minorities, and Sri Lanka - focus on Tamils and Muslims. 


Minority Rights Group, with Grand Synergy Development Initiative (GSDI), Verite Research Pvt Limited, Bytes for All, “Diversity: Impact on 
Vaccine Equality (DIVE).” 


Oklahoma State University, with the MESA Group, “Modeling Vaccine Confidence Interventions for Marginalized Migrant Communities: A 
Mixed Method Approach to Leveraging Social Media Narratives.” 


A study conducted by the Indiana University Observatory 
on Social Media pointed to the potential and need for 
grassroots online campaigns to reach young urban 
Black Americans: 


“Participants would be most likely to share 
messages that have content related to personal 
experiences of ordinary people about vaccines. 
Interestingly, the least shareable messages 
contained images designed to provoke fear of the 
consequences of COVID-19 infection. We also 
tested neutral messages with simple pictures 

of vaccines, and interactive messages that were 
designed to stimulate discussion about vaccines. 
A second part of our study looked at other factors 
why people might be willing to share vaccine-pos- 
itive messages. Unsurprisingly, we found that 
individuals with higher levels of vaccine confidence 
are more likely to share our messages. However, 
when we account for vaccine confidence, we also 
found that people with more open-minded thinking 
styles and more confidence in social media are 
more likely to share our messages. We also studied 
focus group data, provided by our partner, The New 
George Project, to try to understand how best to 
design campaigns to promote vaccines among 
Black Americans. We found three factors related 

to low vaccination rates among Black people: (1) 
concern about safety and efficacy of the shot; (2) 
distrust of the public health system, coupled with; 
(3) fatigue from exposure to its campaigns.” 


21 
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wà Center for Media Engagement ,,, x 


Sponsored 


Black Americans are urged to get vaccinated to 
prevent the spread of diseases. This is an act of 
loyalty to your community. 


SHAREVERIFIED.COM 
Defend Our Community LEARN MORE 
Together 

ny) Like Q Comment a Share 


A COMMUNITY LOYALTY 


For public-health campaigns to be effective 
in driving demonstratable reductions in 
vaccine hesitancy among Black people, 
social media messaging should be framed 
in a way that addresses variations in moral 
values, culture, and communities. Center 
for Media Engagement at the University of 
Austin Texas 


The Indiana University Observatory on Social Media, “The Black and Thriving Project”: The use of message framing and social contagion 
to promote vaccines in African American communities.” 
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VACCINE HESITANT POPULATIONS 


Several studies focused on various types of vaccine hesitant populations. Overall, while the groups 
shared higher levels of hesitancy, the reasons for hesitancy and their views on the vaccine differed quite 
significantly by geographic area, culture, and demographics. 


e |n a study looking at hesitancy by zip code in the US conducted by the University of Washington, 
Institute for Health Metrics and Evaluation, researchers found that there was a stark difference 
demographically for those whose reason for hesitancy was that they felt there was no need for the 
COVID-19 vaccine: “Non-Hispanic White males aged 18-24 had a 46% probability of selecting this 
answer while Non-Hispanic and Asian females aged 35-44, in contrast, only had a 12% probability.’ 
Another study, looking at intervention via small private social media groups of unvaccinated 
individuals in the US, found that communicating in concrete terms the “bottom-line meaning of 
COVID infection, i.e., the serious health risks posed by COVID” was potentially more effective than 
discussing terms in the abstract.” 


e Meanwhile, a randomized trial study 


looking at unvaccinated and vaccine ees ave 


hesitant populations in Japan, conducted a 
by Corowa-kun, also found that while both | am checking in to report that | made the choice to get vaccinated. 
It was hard at first to process, but after having my questions 


interactive and live question and answer answered and feeling it is better to be safe than sorry (thanks to this 
webinars — provided by health care profes- group) | went in today and received my first Moderna shot. | am 


; B : . hopeful that | will be okay with any (if any at all) side effects from 
sionals and chatbots — had little impact the vaccine. | am drinking a lot of water and taking Tylenol as well, 


on vaccine intention, the webinars that so fingers crossed, all will be well. Thank you to everyone here!!! 
allowed participants to engage and have 
their specific questions answered hada O Visit the COVID-19 Information Centre for vaccine x 


resources, 


greater impact and increased the Vaccine si dated 


Confidence Index.**The study saw an 
increase in the importance and the effec- 
tiveness of COVID-19 vaccines significantly 
increased with the webinar intervention. A. CHOOSING TO GET VACCINATED 

Using the Vaccine Confidence Index *° the “This post from a participant in trial 1 indicates that the 
study examined importance of COVID-19 participant chose to get vaccinated directly as a result 
vaccine (2.2 in the control group, -0.8 ee ee a tee 

in the chatbot group, 8.7 in the webinar 

group) and effectiveness of the COVID-19 

vaccine (-8.1 in the control group, -3.4 in 

the chatbot group, 5.3 in the webinar group). 


OO You and 4 others 1 comment Seen by 12 


22 University of Washington, Institute for Health Metrics and Evaluation, “Using social media to identify drivers of COVID-19 vaccine hesitancy 
in the US by county and ZIP code.” 


23 The George Washington University, “Empathic Engagement with the Vaccine Hesitant in Online Spaces.” 


24 General Incorporated Association of Corowa-kun Supporters, “Impact of a LINE chatbot and a webinar on COVID-19 vaccine intention and 
vaccine confidence: a Randomized Controlled Trial among Japanese.” 


25 General Incorporated Association of Corowa-kun Supporters, “Impact of a LINE chatbot and a webinar on COVID-19 vaccine intention 
and vaccine confidence: a Randomized Controlled Trial among Japanese.” “(Changes in VCI were assessed by mixed-effects logistic 
regression model. the difference in % (post-intervention “ pre-intervention) of those who have responded strongly agree or tend to agree) 


” 


e Another study, still underway, conducted 
by The George Washington University, 
also saw preliminary data suggesting that 
“sharing information about vaccines with 
unvaccinated individuals in private Facebook 
groups may improve vaccine confidence 
and may be sufficient to convince a small 
number to vaccinate, even among some of 
the most resistant individuals.” They did note 
that widespread distrust remained a barrier 
to vaccination and issues such as personal 
experience, and pandemic fatigue, were 
among reasons noted to refuse vaccination.’ 


PARENTS AND CHILDREN 


While adolescents and parents recognize the 
importance of vaccines for teens, COVID-19 has 
increased anxiety and impacted parents’ vaccine 
confidence. Studies found that engaging parents 
(through interviews, bulletin board discussions, 
and focus groups), and especially children and 
teens, was an effective way to share information 
encouraging greater confidence in vaccines, in 
some cases leading to uptake. 


e In a study conducted by Unity Consortium”, 
parents reported feeling anxious and uncer- 
tain about so much contradicting vaccine 
information and were unsure who to trust. The 
study also noted that given the controversies 
and polarization around the topic of COVID-19 
vaccination they were more likely to share 
posts from unbiased sources in smaller, more 
familiar social groups, rather than public 
groups where they felt the subject was taboo 
and feared backlash. Parents also appreciated 
straight forward and relevant communication, 
as some have become wary of sources 
with potential hidden agendas, so sources 
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highlighting their unbiased nature were seen 
as more credible. As with many studies, the 
more relevant to their specific situation the 
better. 


e In another study, conducted by Shujaaz Inc.?8, 
Natural Language Processing (NLP) and 
sentiment analysis was used with a qualitative 
exploration of the conversations to compare 
changes to vaccine sentiment and inspire 
dynamic conversations on vaccines. The 
project found that using explicit and relevant 
words was able to drive more focused and 
thematic conversations. It also found that 
a mix of real-life and fictional stories helped 
Shujaaz fans discuss sensitive topics. The 
project saw self-reported data that participants 
not only shared information with friends and 
family, but that they also were inspired to get 
vaccinated. 


PREGNANT AND LACTATING WOMEN 


Vaccine concerns often remain higher than that 
of the average population in many countries for 
pregnant and lactating women. Fears that the 
vaccine may have a negative effect on the mother 
and child are a key source of hesitancy. Several 
studies, not focused on this population, also 
noted lower levels of trust or higher prevalence 

of fertility related rumors impacting hesitancy.?? 
Engaging mothers as well as communities, and 
sharing information in ways that helped to answer 
culturally specific questions, while addressing 
local rumors or fears seemed to prove effective in 
increasing confidence. 


e Astudy conducted by the University of 
California, San Francisco% found Chatbots in 
WhatsApp groups are feasible, acceptable, and 


26 The George Washington University, “Empathic Engagement with the Vaccine Hesitant in Online Spaces.” 


27 Unity Consortium, “Activating Vaccine Supportive Parents of Adolescents and Young Adults as Trusted Voices on Social Media” 


28 Shujaaz Inc., “Tunajengana: Using social media to mobilise young East Africans to champion community uptake of COVID-19 


vaccines” 


29 Grameen Foundation India Private Limited, “D-VACSI: Driving Vaccination Confidence through Social Media Innovations.” 


30 University of California, San Francisco, “WhatsApp Bots for Overcoming Vaccine Misinformation and Increasing Vaccine Confidence 
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31 
32 


may effectively address COVID-19 vaccination 
concerns among pregnant and breastfeeding 
women in rural North India. Despite most 
pregnant and breastfeeding women wanting 
to be vaccinated in this community, COVID-19 
vaccinations remain low. The study identified 
several key barriers, including “fetal/infant 
safety concerns, health provider refusal, and 
gender.’ The chatbot provided a simple yet 
effective way to overcome gender related 
barriers and provide clear and specific infor- 
mation. The study also incorporated ongoing 
feedback, including new answers to questions 
that remained unanswered. While overall 
changes in vaccine uptake and confidence 
were not high, 88.9% indicated the chatbot im- 
proved their COVID-19 knowledge a lot, 71.7% 
recommended it to others, and 88.3% shared 
information from the chatbot with others. This 
highlights the feasibility of using chatbots as 
a resource for a population which often faces 
gender-related barriers to health information 
and care. “Chatbots held potential for impact 
across social networks; this tool could be 
used to help empower women to educate 


families, including household decision-makers. 


Chatbots, linked to trusted sources, expanded 
community health education.” 


e Another study, still underway, looked at 
pregnant and lactating women in Europe and 
found “40% of messages about COVID-19 
vaccination and pregnancy found on the 
investigated social platforms are hesitant 
towards vaccines. According to literature and 
our field work, the main concern about getting 
vaccinated during pregnancy (or shortly 
after/before) is whether the vaccine is safe. 
Concretely, people worry about whether the 
vaccine damages mother and/or child, distorts 
the menstruation cycle or lactation, alters 
the child's DNA or causes disease or even 
death. Vaccine-skeptics also believe that the 
development period of the COVID-19 vaccines 
was too short and that there is a lack of tests 
on pregnant women and their children that 
investigate the long-term effects."*! 


OLDER ADULTS 


While social media is considered by many to be 
something for the young, one of the VCF studies 
showed that social media campaigns targeting 
older adults in lower income areas had “significant 
potential to increase vaccine uptake — especially 
for less well-known vaccines.” “The study, 
conducted focusing on Black populations in the 
UK, noted that this may be an area of underin- 
vestment. It also showed that addressing some 
simple barriers to uptake had significant impact. 
The campaign revealed a number of quick-win 
opportunities for health practices/systems to 
address common barriers to uptake among older 
users in deprived communities.” These quick wins 
include targeting older adults with social media 
and ensuring that booking appointments to see 
GPs or vaccination appointments are easy. 


among Pregnant Women in Northern India: Development and Impact Assessment.” 


University of Antwerp, “Research to increase confidence in maternal vaccination by digital communication interventions.” 


International Longevity Centre — UK, “Leveraging intergenerational relations to increase uptake of immunization in older Black 


communities.” 
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PEOPLE WITH DISABILITIES 


It is important that greater considerations are given to people with disabilities - who often face addition- 
al barriers to access and often more complex health concerns — when it comes to vaccination. 


e A study conducted by RMIT University? found = 
that there is often a significant barrier to vacci- 
nations and vaccine information for people with 
disabilities in Vietnam and Indonesia. Greater 
accessibility to both information and to vacci- 
nation sites, as well as more specific guidance 
on the side effects and safety for those with 
disabilities were some of the key needs the 
study identified. The information that was 
available, was not accessible, particularly to the 
hearing and sight impaired communities. The 
study also found that in Vietnam, Facebook, 
and Zalo (a messaging app and second most 
used social media platform in Vietnam) were 
among the key platforms hearing impaired 
individuals “perceived to be useful for seeking 
vaccine-related information from other people 
with disabilities (PwDs). Those who live in rural 
areas or locations with an unstable Internet 
connection relied on opinion leaders and 
traditional communication means.” Other 
issues were lack of ramps, inaccessible sig- 


GOOD TO GO 
nage, and a lack of sign language interpreters. A person with disabilities getting vaccinated. 


i sac e RMIT University 
Noting pre-existing conditions was not often 


supported on the forms and was also difficult 

to communicate due to a lack of sign language 

interpretation. At the local level, support was lacking, despite the fact that people with disabilities 
were considered part of the priority list in the national vaccination program. The study benefitted 
from strong engagement with people with disabilities, from initial design to implementation, and also 
built on existing local organization support rather than creating new infrastructure. This approach, 
the study recommends, is one governments should ideally pursue and is part of their policy brief 
laying out an accessible approach based on inclusive rights during a pandemic. 


33 RMIT University, with Independent Living Center, funded by SSRC USA, “Modeling vaccine-related information to create hybrid information 
hubs for people with disabilities in Vietnam and Indonesia.” 
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METHODOLOGICAL FINDINGS 


RHETORICAL STRATEGIES 


Many of the studies examined how individuals’ trust were impacted by language, cultural considerations, and 
the way in which information is shared and presented. One project looked instead at how rhetoric around a 
public health institution-the CDC- impacted trust in that institution. 


“The preliminary results of this project highlight the potentially outsized “Engaging with 
role that rhetoric related to the politicization of the CDC may have played a concern- 

in driving CDC trustworthiness and credibility concerns, even relative to eliciting chatbot 
other concerns such as CDC ties to pharmaceutical companies, which are leads Facebook 
commonly cited as drivers of CDC distrust. Further research is required to users in Kenya 


elucidate causal pathways linking politicizing rhetoric to public distrust of the and Nigeria 
CDC; however, the large relative prevalence of this type of rhetoric on Twitter to be more 
seen in this analysis suggests that public health institutions would be wise likely t i 

i i ; i Ea y to repor 
to think carefully about how it can communicate with the public in a way that illi i t 
affirms that its public health recommendations are based purely on scientific ie HIGHT id 
evidence as opposed to political concerns. More broadly, these findings add the vaccine if it 
to increasing concerns that worsening political polarization is threatening the were available 
ability of public institutions to effectively guide and govern the public through today, and more 


crises.*4 The first aim of this project was to shed light on public beliefs about likely to report 

trust in the CDC by evaluating rhetoric related to trust in the CDC found on higher levels 

Twitter during the COVID-19 epidemic. — the study applied an unsupervised, of enthusiasm 

machine learning-based topic modeling algorithm to Twitter data in order to about getting 

categorize the types of rhetoric related to trust in the CDC that Twitter users vaccinated, 

are posting and being exposed to. Results show that rhetorical categories as compared 

both undermining and bolstering trust in the CDC appear to fall in line with to users ina 

prevailing theories about the major components of perceptions of credibility public service 

and trustworthiness, which include beliefs that an entity has the (1) necessary announcement 

competence and expertise, and (2) benevolent and ethical intentions to ahalbotenrcanwel 

behave in a way that meets the normative expectations placed upon it.” aa 

condition. 

CHATBOTS University of Chicago 


Exploring the role of chatbots to share information and answer questions, several 

studies found promising results in their ability to increase vaccine confidence and 

contribute to uptake. This was particularly the case when there was strong community engagement to under- 
stand the underlying factors and concerns, as well as to continuously update the Chatbots to be able to respond 
to the latest trends and concerns. 


34 Jonathan Yong Lee, Stanford University, “Evaluating the Frequency and Effectiveness of Common Rhetorical Strategies Used to Attack and 
Defend the Trustworthiness of Public Health Institutions.” 


VCF IMPACT REPORT | SUMMARY FINDINGS | 27 


e A study conducted by the Laboratory of Data Discovery for Health (D24H) in Hong Kong, Singapore, 
and Thailand, found that “the chatbot was able to disseminate accurate COVID-19 vaccine-related 
information to the public in an easily accessible format from a credible source. Chatbots allowed 
instantaneous responses to pressing questions, which might not be feasible with other forms of 
health communication.” Also, “vaccine chatbots could complement existing tools and offer an in- 
valuable information source that can supplement other forms of health communication and engage 
users who might otherwise feel judged when asking vaccine-related questions. Additionally, social 
listening is critical for keeping a finger on the pulse of real-time changes in sentiment, concerns, 
rumors, and misinformation, as this monitoring allows chatbots to evolve and remain relevant.” 


ARTIFICIAL INTELLIGENCE & MACHINE LEARNING 


Several studies made use of artificial intelligence or machine learning to evaluate their role in increasing 
vaccine confidence via social media. This showed some promising results and encouragement to 
further exploration around the role that this technology might play. 


e A study examining social media communicators “should test artificial intelligence (Al). Our quasi-ex- 
perimental study found that messaging deployed on Facebook that has been enhanced using Al 
can outperform human-generated communication. 
The results from our pilot study found that there was 
a Statistically significantly difference in impact of 
click-through-rate in the Al-mediated communication 
(Al-MC), which means that as more people were 
shown the ads with AI-MC, more people behaved 
with our goal in mind to click through and “learn 
more.” Machine learning-based topic modeling tech- 


Jadufinfisuse / that We've been waiting 
for 


nology has made incredible progress in recent years, wedanfierdiirSadu Qvestons shout vaccines 

and over the course of this project, various older and Pee ee ee SEO k 

newer algorithms were trialed, including latent dir- er . 

ichelt analysis (LDA) and the use of jointly embedded @ ican mee 


topic, document, and word vectors. In the end, the 
use of jointly embedded vectors appeared to strike a 
better balance between having too many topics with 


Questions about vaccines 


adniinonviesw 


sufficient specificity vs. too few topics with diluted/ unimini jene 
; ; se in 196 awnsmäaniayañiaiuiaiu f msy sosse 
insufficient specificity. AMUGDINITNTW LABAN AAMT brons osion 


nineswuiluiasy Becoming vaccine 


Jadu uud? is vaccine good? 
QUESTIONS ABOUT VACCINES [> 
Thailand Chatbot Platform. Laboratory of Data 


Discovery for Health (D24H) : fe? CED 
s vaccine sa 


© Jadu isovaansin lyn |S vaccine safe? 


35 Laboratory of Data Discovery for Health (D24H), “Development and impact assessment of a conversational Al service (chatbot) on 
Covid-19 vaccine confidence and uptake.” 


36 Jonathan Yong Lee, Stanford University, “Evaluating the Frequency and Effectiveness of Common Rhetorical Strategies Used to Attack and 
Defend the Trustworthiness of Public Health Institutions.” 
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e A study conducted by the Center for Media Engagement at the University of Texas at Austin found 
that Al enhanced media outperformed human generated communication. social media communica- 
tors should test artificial intelligence (Al). “Our quasi-experimental study found that messaging 
deployed on Facebook that has been enhanced using Al can outperform human-generated commu- 
nication. The results from our pilot study found that there was a statistically significantly difference 
in impact of click-through-rate in the Al-mediated communication (Al-MC), which means that as 
more people were shown the ads with AI-MC, more people behaved with our goal in mind to click 
through and “learn more."’’ 


HUMAN GENERATED AI-GENERATED 


When you get the COVID-19 vaccine, you help care for 
GARE other people, keep them safe, and prevent others from 
being harmed. 


Black Americans are vaccinated to prevent diseases. This 
is an act of love and care for others in your community. 


When you are vaccinated, you unite with others and join Black Americans are urged to get vaccinated to prevent 


PONAT forces against COVID-19 and defend our community the spread of diseases. This is an act of loyalty to your 
members from getting the disease. community. 
When you get the COVID-19 vaccine, you slow the Black Americans are urged to get vaccinated to prevent 
DECON ON spread of germs, stop contamination, and help prevent the spread of diseases. This is an act of purity and 
infection. keeping your community healthy. 


HUMAN VS. MACHINE 


Al content outperformed human generated content. 
Center for Media Engagement at the University of Texas at Austin 


e Another study conducted by 19 to Zero, looking at COVID-19 vaccine rhetoric on Twitter in hesitant 
marginalized communities found that “(a) mixed-method approach of social network analysis and 
machine learning methods were applied to study COVID-19 vaccine rhetoric on Twitter. Machine 
learning models identified vaccine confident and hesitant tweets within marginalized communities 
(Black, Indigenous, Latinx, and LGBTQ+) in North America. This data was analyzed to elicit themes 
of discussion, emotions, and sentiments surrounding COVID-19 vaccines.... This work highlights how 
machine learning can identify social media trends in vaccines and improve efforts to drive vaccine 
confidence and access." 


37 UAT 


38 19 to Zero Inc., “Novel Use of Natural Language Processing to Analyze Vaccine Attitudes on Social Media and Create Vaccine Confidence 
Messaging for Vulnerable Communities.” 
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CROWD SOURCING 


e One study’’, looking at the impact of crowdsourcing, found that it was challenging, but possible, 
to generate local science-based media content to address vaccine hesitancy and misinformation. 
The study found that of “4,400 social media users who submitted a test video, around 450 people 
uploaded a persuasive video to help get people vaccinated.” While video content improves when 
instructional infographics are provided, “people who submit videos come up with creative and 
intuitive strategies to persuade people that are consistent with behavioral theory, for example, 
highlighting social norms, downplaying risks, emphasizing social benefits and economic effects, 
peace of mind and stress reduction. Storytelling (e.g., people who were affected by Covid) was also 
used as acommon motivator.” The study did find that the success of the campaigns and sourcing 
“local content is possible but challenging and varies significantly by context. To further test the utility 
of crowdsourcing requires resources, time, and patience for experimentation and adaptation. We 
had to play around with ad spend, incentives, and treatment instructions before seeing outputs in 
terms of videos and surveys. This exploratory process was repeated in each country’s context to 
varying results. A comparable small-scale campaign run across countries was more successful in 
Kenya and the Philippines, and less so in Nepal.” 
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A CROWD SOURCED MESSAGES 
Crowd Sourcing Campaign for Facebook in Kenya, Philippines and Nepal. Busara Center for Behavioral 
Economics. 


39 Busara Center for Behavioral Economics, “Crowdsourcing content to combat misinformation in the Global South.” 
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PHYGITAL VS. DIGITAL 


A study conducted by Grameen found “(s)ocial Media experiment leads to 7.6% 
lesser vaccine hesitancy and 1.7% higher vaccine adoption in rural India. The 
study explored “the impact across the digital and phygital arms and revealed that 
Digital-model was more efficient and impactful than Phygital-model in resolving 
information asymmetry, vaccine hesitancy and vaccination confidence. The digital 
model was able to reduce greater proportion of vaccine hesitant persons (by 17%) 
than the phygital beneficiaries (16%). Similarly, the incremental increase in vaccine 


Social Media 
experiment 
leads to 7.6% 
lesser vaccine 
hesitancy and 
1.7% higher 


uptake in the digital model was by 2%. Moreover, digital-model required fewer vaccine 
resources to implement since it eliminated physical contact requirements and is adoption in 
more convenient and scalable. However, low-penetration of internet and internet- rural India 


enabled devices is a challenge that is being addressed by the government's 
telecom policies at a steadily fast pace.”*° 


SUCCESSFULLY REACHING POPULATIONS 


While social media can provide an exciting new source of information for public health, barriers still 
exist, such as internet penetration or the inclusion of community specific languages and considerations. 


One grantee, Institute for Global Health Sciences, UCSF, explored the impact of indigenous languag- 
es on confidence and uptake. They “first conducted a Brand Lift study which showed that videos in 
Mayan languages appear to improve the perceived safety of COVID-19 vaccines, while Spanish vid- 
eos may have improved perceived acceptability of COVID-19 vaccines for Spanish speakers. Finally, 
we conducted an evaluation using in-person surveys with members of indigenous communities 
before and after deploying the social media video campaign and found increased odds (1.83 times 
the odds) of getting vaccinated among those who saw our videos over social media compared to 
those who did not, adjusted by age, community, sex and language spoken at home. Findings from 
this study can help key stakeholders at the national and local level consider optimal practices for 
development of vaccine education content and how to leverage this content over social media to 
increase COVID-19 vaccine uptake in other regions of the country...” We completed 1,681 pre and 
post intervention surveys, the mean age of participants was 32 years, 63% (N=998) identified as 
women and 36% spoke an indigenous language (K’iche or Kaqchikel) at home. Twenty-one percent 
of the participants (N=327) reported having watched the pre-created content. From those who saw 
the videos 98% reported to have learnt something new about the vaccines from them. At baseline, 
89% (N=1402) of participants reported having been vaccinated against COVID-19 with at least one 
dose compared to 97% (N=1507) in the follow-up.“ 


40 Grameen Foundation India Private Limited, “D-VACSI: Driving Vaccination Confidence through Social Media Innovations.” 


Institute for Global Health Sciences, UCSF, “Who to trust: combating COVID-19 vaccine misinformation through trusted messengers and 
social networks in indigenous communities in Guatemala.” 
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e Another study, conducted by Purpose 
Campaigns LLC, demonstrated that 
messaging can be shared across plat- 
forms, increasing reach and while yield 


Protect Your 


rates were relatively low, using automa- Community: 


tion had the potential to increase ease 


of access and reach. “Open WhatsApp Get Vaccinated f 
Against 


group channels provide rich territories 
for seeding pro-vaccine content. 28% 
of groups engaged with a pro-vaccine ° 
content link. Over 5% of the content Covid- 19 
links initiated in public group channels 
were re-shared within WhatsApp to 
other WhatsApp channels one or more 
times. Seeding content into WhatsApp 
can generate clicks and shares to 
public social media platforms such as 
Facebook. A high volume of WhatsApp 
users can be reached at scale. Our re- 
search teams were able to identify over A CONTENT WITH CTA 
fifty thousand open WhatsApp group Automation of messages like this one generated by 
channels in four specific countries, tar- Purpose Campaigns was found to increase ease of 

‘ . access and reach. 
geting groups associated with a range 
of specific interests from politics to football and alumni groups that all provide opportunities to both 
learn about and develop content for audiences with clear associative ties. Nevertheless, the process 
of joining these groups is time consuming and challenging. Yield rates of viable groups are relatively 
low (roughly 1%) and manual group joining can be cumbersome. However, automation tools can 
provide capacity to reach large numbers of WhatsApp users at scale by automating the group 
joining and evaluation process. At least 20 percent of links that were initially shared on WhatsApp 
via our researchers appear to have been re-shared to other WhatsApp groups or individuals. In India, 
9.8% of links were reshared, in Nigeria 4.7% were reshared and in South Africa 31.8% were reshared. 
These do not include additional sharing via in app forwards, which could not be directly observed.”*? 


42 Purpose Campaigns LLC, “Harnessing virality for vaccine confidence: testing tactics to optimize sharing of pro vaccine content between 
WhatsApp and traditional social media platforms.” 
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ENGAGING TRUSTED VOICES 


Anyone looking to promote vaccine confidence within Indigenous communities in Canada should first 
look to support self-determination and community-driven solutions. Communities have expertise and 
local cultural and social knowledge that are crucial for effective vaccination promotion. 


Supporting community, however, is not the first step in promoting Indigenous vaccine confidence. 
Before being able to support community, mutually beneficial and authentic relationships must exist 
between community members and healthcare workers or health researchers. Forming and strengthen- 
ing these relationships should be a priority so that they are in place before the next major public health 
crisis emerges.*? 


e A study conducted by the University of North Carolina at Chapel Hill explored the impact of trusted 
political voices by testing the impact of sharing positive messages by trusted leaders, in this case 
Donald Trump, in over 1,000 low vaccination counties along with another set of 1,000 plus similar 
counties that would not receive it. They saw an increase in vaccinations in those who saw the PSA. 
“The average county that got the ad recorded an increase of 103 vaccinations more than the coun- 
ties we did not treat. With over 1000 counties getting the Trump endorsement, that translates into 
better than 100,000 more vaccines. Our ad budget was slightly less than $100,000 dollars, which 
means that the cost of each new vaccine was less than a dollar. That is far more efficient than other 
efforts that states tried to increase vaccine uptake such as lotteries.” 


“Hesitancy in the U.S. proved especially high, with rates of vaccination barely among the top 50 
countries in February 2022 (25). As many have observed, a tragedy of the COVID-19 pandemic 
is the extent to which protective measures became tangled in Americans’ political identities, 
which led to deaths and suffering that could have been avoided. But if politics characterizes one 
aspect of the problem, it might also point to part of a solution. We find that positioning Donald 
Trump and Fox News as counterintuitive messenger is a cost-effective way to overcome 
hesitancy among people who still had not been vaccinated, months after the vaccines became 
widely available. Are our findings scalable? We believe they are. As long as Americans on the 
political right are a significant bastion of hesitancy, support from Donald Trump and other party 
leaders for vaccinations will represent a potent tool that public health messengers can use. 
Leveraging his recent endorsement of COVID-19 boosters is an obvious extension of this work. 
So, too, with seasonal flu vaccines and immunization attitudes generally, which are starting to 
show signs of increased partisan schism (6). Insofar as vaccinations continue to be politicized, 
this research provides a model for political messaging as an important public health tool in 
future pandemics. Whether the dividing line is politics or something else, we encourage public 
health proponents to reflect on messengers whose voices might carry special weight among 
target populations. For example, other research finds a pro-masking message from a military 
general increases support for masking among political conservatives (26). The results we 
report herein help corroborate this theme. We think it represents a promising route to overcome 
resistance and, in turn, save lives.” 


43 Morning Star Lodge, “Where Past Meets Present: Understanding COVID-19 Vaccine Hesitancy among Indigenous Peoples in 
Saskatchewan.’ 


44 University of North Carolina at Chapel Hill, Rahsaan Daniel Maxwell, “Targeted Vaccine PSAs Based on Political Science and Economic 
Insights.” 


45 University of North Carolina at Chapel Hill. (6) A. Fridman, R. Gershon, A. Gneezy, COVID-19 and vaccine hesitancy: A longitudinal study. 
PLoS ONE 16(4): €0250123, (26) S. Greene, M.J. Hetherington, R. Maxwell, T.J. Ryan, Public service announcements and promoting face 
masks during the COVID-19 pandemic, PS Polit. Sci. Polit., 55, 7-12 (2021) 


BEHAVIORAL BARRIERS 


A study conducted by the Behavioral Insights 
Team found three key behavioral barriers that 
might impact uptake: 1) pro-vaccine messag- 
ing may work better when risks are perceived 
to be higher; 2) friction costs, small details that 
can make vaccination appear more effortful, 
may impact the likelihood of getting a shot; 
and 3) research showed that people are good 
at visualizing goals, but not the steps it takes 
to achieve them.*° 


BEHAVIORAL NUDGE 


A project conducted by Mali Health had 
significant impact in increasing confidence 
in peri-urban communities in Bamako, Mali 
through participatory methods and tools to 
develop and disseminate voice-based social 
media messages to improve vaccination 
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knowledge and confidence.*”’ The project 
incorporated end-users in every step of the 
program, from developing the platform to 
understanding the needs and behaviors of 

the target population and the norms which 
govern them. While 95% of women reported 
not trusting COVID-19 vaccines at the start 

of the project, 100% demonstrated improved 
knowledge about the benefit of vaccination, 
75% of the women who used the application 
expressed confidence in vaccination against 
COVID-19, and 60% felt better equipped to 
convince others to get vaccinated against 
COVID-19. The primary concern, at the start of 
the project, was a lack of trust in the COVID-19 
vaccine and the lack of access to trusted 
health personnel to address their concerns. 
The Kéneya Blon application was able to 
remove barriers by providing access to health 
care personal to address their concerns. 
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A EFFECTIVE NUDGES 


A graphic looking at how vaccine positive pathways could be more successful at meeting the demand for information. 


16 The Behavioural Insights Team, “Designing and testing a WhatsApp chatbot to increase Covid-19 vaccination bookings in Argentina.” 
17 


Mali Health, “Use of voice-based social media messages to improve vaccination knowledge and confidence in underserved peri-urban 


communities.” 
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e Another study conducted by Upstream Thinking, LLC, looking at identifying online behavioral nudge 
interventions, also noted the unexpected and at times inconsistent behavior in unvaccinated African 
American populations in the rural South.*8 In contrast to the findings noted above by Mali Health, 
Upstream Thinking, LLC, found that their target population was less inclined to defer to the decisions 
of medical and public health experts, and instead motivated to arrive at their own conclusions. This 
is in contrast to a historical precedent of vaccination, where trust in doctors and uptake of vacci- 
nations was more widespread in this community. The study noted five key drivers of demand that 
can be leveraged to direct demand towards vaccine positive interpretations. These key drivers were 
individuals seeking balance and diversity in narratives, freedom to choose (looking for information 
rather than instruction), trust (from firsthand sources rather than from institutions), transparency, 
and certainty about value (not spending time on information that does not fit their needs). When 
social media concepts were aligned with these needs, the quantitative results indicate a nudge 
towards more vaccine positive content. 
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Á SOLUTION CONCEPTS AND RESULTS 
Screen captures from YouTube demonstrating the various stimulus. 


48 Upstream Thinking, LLC, with Final Mile Consulting, “Redirecting existing demand for information towards vaccine confident interpretation 
by identifying online behavioral nudge interventions.” 


PROCEDURAL INSIGHTS 
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A key purpose of the Fund was to identify and share learnings from the process side from those who 
were conducting the research to help expedite and hone future projects. The procedural insights 
detailed below are some of the common learnings from research teams. For more detailed insights see 


the individual grantee overviews. 


COLLABORATING WITH TRUSTED 
SOURCES 


Whether engaging with local organizations that 
already have connections in the community, or 
using familiar platforms, researchers found they 
had greater success in engaging communities 
when community representatives were on the 
research team. This was particularly relevant given 
the expedited timeframe of the research. Several 
studies also noted that participants were more 
receptive to information with transparent sources. 
Unbiased information was seen as particularly 
valuable, as hidden agendas were a common 
concern and cause for distrust. 


“Communities are dynamic, and this includes their 
priorities. Something identified as high priority 
when funding is applied for may not remain so 
when funding is approved, and research begins. 
Furthermore, the piloting of any solutions must 
include what community intends to do. Many have 
argued that the solutions to Indigenous health 
inequities lie within Indigenous communities, 
Knowledges, and cultures. Therefore, funding 
proposals for health research with Indigenous 
communities should be written such that research 
can pivot from seeking for solutions, to supporting 
and seeking to understand Indigenous-led 
self-determined solutions”. “Indigenous vaccine 
hesitancy does not necessarily mean low vacci- 
nation rates. Assuming that Indigenous vaccine 
hesitancy will result in low vaccination rates is defi- 
cit-based and undermines the strengths, abilities, 
and expertise within a community. In the complex 
space of Indigenous vaccine confidence, many 


factors, such as culture, can be used to overcome 
vaccine hesitancy or justify refusal. Furthermore, 
the community members engaged in this research 
do not perceive empathy to be a consistent 
component of vaccine confidence promotion. 
Vaccination-promoting social media posts that are 
entirely developed by community appear to per- 
form better than those that are, partially or entirely, 
developed externally. While the sample size small, 
the six posts developed, and piloted, during this 
research had a mean of 0.(66) N likes and 46.1(66) 
KX views. The six community-developed posts that 
most closely preceded this research had a mean 
of 2.(33) i likes and 64.(66) views.” 


ENGAGE COMMUNITIES EARLY 


An initial phase of engaging communities directly 
and gathering input from the target population 
helped to ensure the later success of the project. 
In particular, this phase helped to avoid the ex- 
tractive nature and ensured that they were relevant 
and took local socio-cultural factors into account. 
Identifying key narratives trusted community 
members or motivations as well as defining specif- 
ic concerns and fears around vaccination were key 
in many projects. It also helped to identify specific 
unmet needs or barriers in these communities. 
Collecting this early information also helped 
researchers to better asses how the interventions 
were received. “Building trusting relationships 

and encouraging stakeholders’ ownership 

over a project early on is crucial for successful 
collaboration.” 


49 Morning Star Lodge, “Where Past Meets Present: Understanding COVID-19 Vaccine Hesitancy among Indigenous Peoples in 


Saskatchewan.” 


50 The Behavioural Insights Team, “Designing and testing a WhatsApp chatbot to increase Covid-19 vaccination bookings in Argentina.” 
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MIXED-METHOD RESEARCH PROVIDED RICH FINDINGS 


Many projects took a mixed method approach, 
combining qualitative interviews, human-cen- 
tered design solution development, quantitative 
solutions, and lever testing. This combination of 
approaches and research provided a much richer 
and holistic view of the challenge of vaccine 
hesitancy." It also allowed researchers to create 
more persuasive narratives and demonstrate the 
effectiveness of identifying culturally situated 
discourses to amplify organic community 
narratives to boost vaccine confidence. 


Another study had significant results in increas- 
ing vaccine uptake through a mixed methods 
approach. “IRD initiated project Boost in three 
high-risk districts in Karachi, Pakistan, employing 
participatory engagement to build user informed 
social media campaigns around COVID-19 
vaccine confidence. From October 2021 to 

June 2022, the project leveraged six online 
platforms (WhatsApp, Facebook, Instagram, 
Twitter, TikTok, and Snack Video) to capture 
diverse audiences from the identified districts 
and ran multiple online campaigns, utilizing 
user-informed content, based on social barriers 
faced by communities during the pandemic. 
The project supplemented these online efforts 
via immersive on-ground activities, including 
Theatre of the Oppressed performances, health 


education, and various Below-The-Line (BTL) 
activations. Six online groups were developed 
(mixed gender on Facebook and women-only 
on WhatsApp) with 1,425 members, stimulating 
dialogue and dissemination. Moreover, the 
on-ground team engaged over 5,700 people 
across the catchment areas through on ground 
sessions and BTL activities. Through these 
efforts, 1,039 people availed COVID-19 vaccines 
through mobile camps held in the selected 
districts. It was noted that a strategy bridging 
online engagement with on-ground activities 
highlighting the efficacy of vaccines to restore 
normalcy in a post-pandemic world garnered 
public interest and uptake of vaccines. Thus, 
participatory approaches embedded in outreach 
efforts ensure inclusion of otherwise neglected 
narratives in-person and through various social 
media.”°? 


“Through these efforts, 1,039 
people availed COVID-19 
vaccines through mobile camps 
held in the selected districts.” 


51 Upstream Thinking, LLC, with Final Mile Consulting, “Redirecting existing demand for information towards vaccine confident 
interpretation by identifying online behavioral nudge interventions,’ and Oklahoma State University, with the MESA Group, “Modeling 
Vaccine Confidence Interventions for Marginalized Migrant Communities: A Mixed Method Approach to Leveraging Social Media 


Narratives.” 


52 IRD Global, “Boost: Building vibrant online communities to support COVID-19 vaccine uptake through participatory engagement 


principles.” 


LOW INTERNET PENETRATION 


While social media is a key tool for those who have 
access to technology and bandwidth, some com- 
munities remain hard to reach. When data in these 
communities were limited, researchers combined 
findings with additional, more traditional sources 
of non-digital information that helped to validate 
or expand findings. As one grantee noted, “though 
access to technology is increasing, it will continue 
to be a limiting factor for millions, especially for 
women with limited literacy and numeracy skills.’° 
“Due to low mobile network coverage and low 
infrastructure development with telecommuni- 
cations in Cameroon, the internet remains very 
expensive and limited so in-person interviews were 
required for data collection. Interview fieldwork 

in Cameroon had to overcome more difficulties 
related to training, research delays, and travel 
restrictions due to safety concerns. The geograph- 
ical characteristics led to long-distance travel for 
the data collection team to reach the selected 
communes in the regions.’ 


GROUP MODERATION 


As a further grantee noted, “moderating large 
groups is a full-time job. Not every comment 
requires a response and may be counterproduc- 
tive.” Several grantees noted the difficulty of 
striking the right balance between listening and 
sharing information, as well as when to ignore or 
even remove participants that may be introducing 
controversial or extreme viewpoints. Bringing in 
a professional facilitator was another approach 
to ensuring conversations were productive and a 
positive experience for participants. 
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ENGAGING AND RECRUITING 
PARTICIPANTS 


The area of health-focused research in social 
media is a fairly new one, and our researchers 
encountered a few issues around social media 
accounts, blocked ads, data protection laws, 
ethics requirements, and navigating recruitment. 
Partnering with an organization that has experi- 
ence in these areas, or creating multi-disciplinary 
teams is a useful approach to be able to include a 
variety of expertise. Reaching the right audience 
within the confines of social media platforms 

and social media advertising was also a learning 
process. For example, focusing on zip codes 
rather than demographics was a necessity given 
the tools available. Actual engagement that was 
relevant and motivating to those suffering from 
“COVID fatigue” was a key factor in participation. 
Finally, relevant narratives and understanding what 
drives local communities was an important way to 
surmount this issue. 


“The importance of collaborative solution design 
should not be underestimated. Co-designing with 
local partners and citizens through joint working 
sessions, focus groups, and user testing, led to 
significant improvements in the chatbot design. 
Using WhatsApp as a means to send official com- 
munications from a Ministry of Health comes with 
the challenge of creating trust in the service. Fear 
of scams and distrust in digital communication 
pose implementation risks that should be mitigat- 
ed.”°° Developing and hosting a WhatsApp chatbot 
using government IT systems may present tech- 
nical challenges and requires local partners to be 
dedicated to the success of the project. 


53 Minority Rights Group, with Grand Synergy Development Initiative (GSDI), Verite Research Pvt Limited, Bytes for All, “Diversity: Impact on 


Vaccine Equality (DIVE).” 


54 Center for Media Engagement at the University of Texas at Austin, “Leveraging moral values and social media to boost vaccine confidence 
and overcome medical mistrust and historical racism among Black populations in the U.S. and Cameroon.” 


55 David A. Broniatowski and Lorien C. Abroms, “Empathetic Engagement with the Vaccine Hesitant in Online Spaces” 


56 The Behavioural Insights Team, “Designing and testing a WhatsApp chatbot to increase Covid-19 vaccination bookings in Argentina.” 
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RESPONDING TO CHANGING CIRCUMSTANCES 


“Given the quickly-changing nature of the pandemic, modifications to local COVID-19 vaccination pro- 
tocols are to be expected and affect intervention design. Staying informed about planned changes and 
allowing for flexibility in the intervention design is key to success.” Furthermore, “People's health-related 
perceptions and behaviors change rapidly as the pandemic evolves. Research and intervention design 
must be updated to the changing circumstances. Using non-standard qualitative research techniques 
(e.g., reviewing comment sections of vaccine clinic Google listings) can generate rapid insights, which 
can later be validated through in-person field research activities.” 
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Á INFLUENCERS INFLUENCING 


Posing at a Shujaaz event for the social media vaccination campaign, 
young influencers show off their vaccination status. 


57 The Behavioural Insights Team, “Designing and testing a WhatsApp chatbot to increase Covid-19 vaccination bookings in Argentina.” 


THE WAY FORWARD 


WHAT CAN STAKEHOLDERS DO? 
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Aligning vaccine interventions with the needs of the vaccine hesitant and unvaccinated is key in 
increasing vaccine confidence and uptake. Governments, health authorities, researchers, and other 
stakeholders can play a key role in increasing vaccine confidence through social media. 


COMMUNITY ENGAGEMENT 


e Involve communities, or local organizations, 
particularly those in hard-to-reach or margin- 
alized communities, at every step, including 
in the study or technology design as well as 
implementation. To be most effective, these 
should be adapted to the realities of each 
community or population it is trying to serve. 
This should include the language and images 
used, but also the social-cultural and gender 
norms, the kinds of misinformation circulating, 
and the specific concerns of the community. 


e Researchers can also pair with policymakers 
from the beginning of their projects to ensure 
that their findings get translated into policy and 
become actionable at the local, state, and if 
appropriate, national level. 


e Partner with local organizations to help plan, 
understand, and implement programming. 


e Consider amplifying existing expressions of 
vaccine confidence from within the community 
as hearing from individuals trusted in the com- 
munity can have a more significant impact. 


PEER CONFIDENCE p 


Targeting Older Generations? Screen capture from 
a social media campaign from the International 
Longevity Center exploring intergenerational 
relations in increasing routine vaccinations. 


LISTENING TO NEEDS 


Provide information to help individuals make 
their own decisions rather than seeking to 
instruct. This can mean sharing positive and 
negatives in an unbiased fashion that is still 
supportive of vaccines. 


Where possible, provide a platform for small 
group dialogues and peer support for targeted 
communities and provide relevant moderation. 


Where distrust exists between local popula- 
tions and health authorities or government, 
extra efforts should be made to involve trusted 
figures to convey key information and rebuild 
trust. 


Seek to understand social and accessibility 
barriers to vaccination. 


Have You Discussed The Jab Yet? 
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GOVERNMENTS 


Governments play a key role in health campaigns, creating access to vaccines and shaping the political 
environment for or against vaccinations. As can be seen in the results from the study conducted by the 
University of North Carolina at Chapel Hill, sharing content with politicians endorsing vaccines resulted 
in an uptick in vaccinations in those counties that received the messaging.°® 


e Another study conducted by Macro-Eyes, Inc., noted the key role that governments and health offi- 
cials can play and the importance of monitoring. “(T)he use of social media to monitor and respond 
to public sentiment around vaccines is a valuable tool for health officials and allows health officials 
to get an accurate picture of public sentiment surrounding vaccines. This information can help 
precision allocation of resources during vaccination campaigns, including how and when to roll out 
vaccinations to specific populations. In addition, we have discovered that using specific platforms 
(such as Facebook) for this purpose is more effective than others (such as Twitter). This information 
could be used by public health officials to develop targeted outreach campaigns in order to improve 
vaccination rates among at-risk populations.” 


“(O)ver 1,000 low vaccination counties to receive the “Trump treatment” along with another set 
of 1,000 plus similar counties that would not receive it. We ran the PSA for three weeks, and 
then compared whether vaccinations increased in the counties that got the Trump vaccine 
endorsement PSA compared with those that did not.” ... “They did. By a lot. The average county 
that got the ad recorded an increase of 103 vaccinations more than the counties we did not 
treat. With over 1000 counties getting the Trump endorsement, that translates into better than 
100,000 more vaccines. Our ad budget was slightly less than $100,000 dollars, which means 
that the cost of each new vaccine was less than a dollar. That is far more efficient than other 
efforts that states tried to increase vaccine uptake such as lotteries.” 


University of North Carolina at Chapel Hill, Rahsaan Daniel Maxwell, 
Targeted Vaccine PSAs Based on Political Science and Economic Insights 


“Effective health communication balances the old and the new: sound conceptual ideas that 
have guided research and practice for many years with rapidly evolving and algorithm-driven 
information landscape that can greatly enhance reach and outreach. Health communicators 
have known for a long time that effective communication requires understanding the audience; 
tailoring messages according to audience members’ needs and expectations; using credible 
and trustworthy sources to disseminate messages, and being sensitive to cultural nuances that 
affect message interpretation. These considerations remain as important as ever.” 


John Hopkins University, A Social Media Approach to Young People 
in India Serving as COVID-19 Vaccine Advocates 


58 University of North Carolina at Chapel Hill, Rahsaan Daniel Maxwell, “Targeted Vaccine PSAs Based on Political Science and Economic 
Insights.” 


59 Macro-Eyes, Inc., “Predicting vaccine hesitancy among eligible communities in Western Cape Province, South Africa.” 


H 


VCF IMPACT REPORT | SUMMARY FINDINGS | 41 


EALTH ORGANIZATIONS 


Health organizations, both governmental, medical or advocacy in nature have a large role to play in vaccina- 
tions. It is clear that the more investment is made in understanding the particular attributes of a community 
and their particular hesitancy, as well as monitoring and adjusting to changes, the more effective and efficient 
the roll out. 


“Learning sentiment through leveraging of public social media data can 
also help identify any areas where there may be opposition or hesitancy 
towards getting vaccinated, so that these issues can be addressed directly 
by health officials. Additionally, tracking social media chatter provides a 
way for health officials to quickly address any concerns or misinformation 
that may arise online about vaccines. Doing so helps ensure that people 
are accurately informed about the safety and benefits of vaccinating 
themselves and their families. Our analysis found that both sentiment 
analysis (measuring public confidence) and spatial accessibility (measur- 
ing ease of access) are important indicators for understanding why some 
communities may be hesitant about getting vaccinated.” 


“A key reason for vaccine hesitancy within Black communities is the 
distrust of medical advice from the government and the public health 
system. Health campaigns in the future should make it a high priority to 
address this issue. Our results indicate that grass-roots campaigns on 
social media have good potential for reaching many Black Americans. 
Those campaigns should target social networks in which Black 
Americans have a strong level of trust. We note that while Twitter is 
currently trusted, this may change over the next year due to its change 
of ownership. We have a number of recommendations about the types 
of content which would work best in grassroots campaigns. 1. We found 
that campaigns that are personally relatable work best. For example, 
those that use images of people and tell personal stories. 2. Avoid 
campaigns intended to leverage fear responses. For instance, images 
that contain portrayals of ill or dead people.”°' 


“It is clear that as policy makers and health institutions look towards A VACCINE PROMOTION VIA 


a TWITTER 
increasing vaccine efforts, it will be important to examine and under- j ae 

; F i ; The Indiana University Observatory 
stand the specific drivers of vaccine hesitancy for each group and on Social Media 


tailor the approaches to address the level of hesitancy and the reason 

for it."°* Providing small online groups or webinars, where concerns can be 

addressed in real time and information is tailored and focus on the bottom line impact of COVID-19, may be 
a useful tool to increase vaccine confidence. Finally, when working to increase vaccine demana, it is vital that 
the supply side is also considered. Increased demand, without increased or adequate supply may do more 
damage by eroding trust in the healthcare system. Creating greater demand increases vaccinations, as long 
as there are vaccines available. 


60 
61 


62 


Macro-Eyes, Inc., “Predicting vaccine hesitancy among eligible communities in Western Cape Province, South Africa.” 


a 


promote vaccines in African American communities.” 


University of Washington, Institute for Health Metrics and Evaluation, “Using social media to identify drivers of COVID-19 vaccine hesitancy in the 
US by county and ZIP code.” 
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NGOS 


For several studies, the role of community organiza- 
tion and local NGOs played a key role as community 
ambassadors. NGOs also can play a key role at a 
local, national, and international level via advocacy 
and engagement to increase vaccine confidence. 


A study conducted by UCSF Guatemala provided 
some very useful recommendations for communi- 
ty organizations: 


1. Community based organizations have 


pre-existing priorities and projects that may 
compete for time and human resources. 
Thus, any project under development must be 
designed to ensure the research, and potential 
implementation project developed from that 
research, serves the organization's purpose 
and mission from the beginning. 


. Research-informed interventions can 


increase activity effectiveness. Research can 
help guide CBOs towards more efficient and 
effective targeting of interventions. 


. Trust is critical. Numerous interviews and 


focus groups indicated that local healthcare 
workers, including community health workers, 
and local councils, were trusted sources of 
information. Understanding local dynamics 
around who is trusted and why, can allow for 
better targeting of resources. Trusted com- 
munity organizations and leaders can likely 
have an outsized impact on lives when trust in 
central government is low. 


. Technology and social media can be leveraged 


to change behaviors. As social media use 
increases in historically marginalized communi- 
ties, it can be used as a platform for community 
organizations and leaders to communicate and 
influence health decision making. 


63 


“The Black and Thriving Project”: The use 

of message framing and social contagion 
to promote vaccines in African American 

communities 


. Community voices MUST be front and central 


in dissemination activities. Ensuring commu- 
nity partners are leading the way on podcasts, 
interviews with press, and are active members 
and authors on journal article writing teams 

is critical for appropriate recognition, capacity 

building, and continuity.”° 


RESEARCHERS 


Invest in the technical infrastructure 
necessary to run complex campaigns and 
multi-stage surveys in developing country 
contexts. The rationale for our target geog- 
raphies was the paucity of rigorous online 
experiments in these contexts. As a result, 
we also had to build the online campaigns 
and experiments infrastructure from scratch 
in parallel with implementing an ambitious 
multi-country study. A key hurdle was figuring 
how to process thousands of small survey 
payments and incentives across geographies 
with different policies in a cost-effective way. 
Investment in infrastructure will continue to 
serve and expand for future studies, but it 
was a big challenge to build out and required 
dedicated engineering, technical resources, 
time, and changes to our study protocol. 


Including studies on specific populations 

and their needs. As recommended by the 
University of Antwerp: “including pregnant and 
lactating women in pre-marketing clinical trials 
should be encouraged since lack of testing 

on this target group is one of the reasons why 
women doubt or refuse to get vaccinated 
during pregnancy or lactation.”©° 


Institute for Global Health Sciences, UCSF, “Who to trust: combating COVID-19 vaccine misinformation through trusted messengers and 


social networks in indigenous communities in Guatemala.” 


64 Busara Center for Behavioral Economics, “Crowdsourcing content to combat misinformation in the Global South.” 


65 University of Antwerp, “Research to increase confidence in maternal vaccination by digital communication interventions.” 
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AREAS OF NEEDED RESEARCH 


VCF represented a first step for much of this research, but further exploration is needed. Advancing 
Health Online (AHO) disseminated a survey to grantees and valued partners to collect external input to 
help sharpen the second research program. Areas of needed research, as identified by survey respon- 
dents, have been detailed below. 


e Two primary research themes emerged when œ Respondents were split roughly 50/50 on 


respondents were asked what research focus 
would be most conducive to advancing AHO’s 
goal of increasing the understanding of the 
impact social media has on health outcomes 
and resilient communities: 


whether the next research program should 
focus on idea generation or implementation 
science, with several emphasizing that there 

is a strong need for both.Respondents agreed 
that the next round of research should support 


closed-loop feedback. To ensure closed-loop 
feedback can be collected and measured, 
participants recommend: 


» Influencing social norms, and 


» Driving community and health care provider 


engagement. 
» Rigid requirement for causal identification 


ny ee ae strategies, 
e Furthermore, two additional underpinning priori- g 


ties were articulated and should be considered » 
across all research: 


Real-time involvement and engagement of 
key stakeholders and experts, 

Agreements with local or central govern- 
ments or entities to integrate or share data 
in order to help validate uptake and/or 
vaccinations from research, and 


» Equity — better target and measure 
impact related to reaching and impacting 
under-served groups and ethnic minorities, 


and » More time to run focus groups before and 


after campaigns, such as representative 
samples, quick surveys to ask about action, 
and better links to book vaccinations. 


» Closed-loop feedback - vaccine message 
exposure should be causally linked with 
actual vaccine uptake. 


PATH FORWARD FOR ADVANCING HEALTH ONLINE 


The Vaccine Confidence Fund was AHO's first program to meet the urgency of the COVID-19 pandem- 
ic. Our goal with the VCF was to help accelerate a body of research and insights that would inform mass 
vaccination campaigns and seek to address vaccine confidence amid the need to vaccinate the global 
population. We were also committed to moving quickly and simultaneously working with key partners to 
disseminate evidence and insights on this topic to ensure vaccination campaigns were more impactful. 
This commitment further assists in increasing vaccine confidence within underserved and underrepre- 
sented communities. 


Looking forward, AHO is announcing a new call to action with another round of funding to support 
research on vaccine confidence with a strong emphasis on routine immunizations. The call for propos- 
als will focus on two target populations, healthcare workers and local communities, and will seek to 
improve understanding on how they can be supported to drive positive health actions via social media 
engagement and build vaccine confidence around COVID-19 vaccines and routine immunizations. 
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Á VACCINATING YOUTH CHAMPIONS 


A vaccination is given by a Kenyan Ministry of 
Health nurse at a vaccination drive by Shujaaz. 


INDIVIDUAL 
GRANTEE REPORTS 


Each grantee was asked to share a high-level 
update on the status of their respective research. 
These updates can be found below. 
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Emphatic Engagement with the Vaccine Hesitant in 
Online Spaces 


48 


19 To Zero Inc. 


Novel Use of Natural Language Processing to Analyze 
Vaccine Attitudes on Social Media and Create Vaccine 
Confidence Messaging for Vulnerable Communities 


51 


Busara Center for Behavioral Economics 


Crowdsourcing content to combat misinformation in the 
Global South 


53 


Center for Media Engagement at the University of Texas 
at Austin 


Leveraging moral values and social media to boost vaccine 
confidence and overcome medical mistrust and historical 
racism among Black populations in the U.S. and Cameroon 


57 


Corowa-kun 


Impact of a LINE chatbot and a webinar on COVID-19 
vaccine intention and vaccine confidence in Japan: a 
Randomized Controlled Trial 


59 


Grameen Foundation India Pvt Limited 


D-VACSI: Driving Vaccination Confidence through Social 
Media Innovations 


62 


Institute for Global Health Sciences, University of 
California, San Francisco 


Who to trust: combating COVID-19 vaccine misinformation 
through trusted messengers and social networks in 
indigenous communities in Guatemala 


64 


Institute of Tropical Medicine Antwerp 


Cultivating Online Safe Spaces: Addressing unspoken 
hesitancy to build vaccine confidence in healthcare 
workers in Belgium 


66 


International Longevity Centre 


Generation Vax — Leveraging intergenerational relations to 
increase vaccination uptake 


68 


IRD Global 


Boost: Building vibrant online communities to support 
COVID-19 vaccine uptake through participatory 
engagement principles 


71 


Johns Hopkins University 


A Social Media Approach to Young People in India Serving 
as COVID-19 Vaccine Advocates 


74 


Jonathan Yong Lee (Individual) - Stanford University 


Evaluating the Frequency and Effectiveness of Common 
Rhetorical Strategies Used to Attack and Defend the 
Trustworthiness of Public Health Institutions 


76 


Laboratory of Data Discovery for Health (D24H) 


Development and impact assessment of a conversational 
Al service (chatbot) on Covid-19 vaccine confidence and 
uptake 


78 


Macro-Eyes, Inc. 


Predicting vaccine hesitancy among eligible communities 
in Western Cape Province, South Africa 


81 


Mali Health 


Use of voice-based social media messages to improve 
vaccination knowledge and confidence in underserved peri- 
urban communities 


83 


Minority Rights Group 
Diversity: Impact on Vaccine Equality (DIVE) 


85 


Morning Star Lodge 


Where Past Meets Present: Understanding COVID-19 
Vaccine Hesitancy among Indigenous Peoples in 
Saskatchewan 


88 


Oklahoma State University 


Modeling Vaccine Confidence Interventions for 
Marginalized Migrant Communities: A Mixed Method 
Approach to Leveraging Social Media Narratives 


90 


ORB International 


Exploring drivers of HPV vaccine confidence and 
community-led solutions leveraging the experiences of 
mothers, daughters, and health-care providers in Brazil 


92 


Purpose Campaigns LLC 


Harnessing virality for vaccine confidence: testing tactics 
to optimize sharing of pro vaccine content between 
WhatsApp and traditional social media platform 


94 


RMIT University 


Modeling vaccine-related information to create hybrid 
information hubs for people with disabilities in Vietnam 
and Indonesia 


97 


Shujaaz Inc. 


TUNAJENGANA: Using Social Media to Mobilize Young 
East Africans to Champion Community Uptake of Covid-19 
Vaccines 


100 


The Behavioural Insights Team 


Designing and testing a WhatsApp chatbot to increase 
Covid-19 vaccination bookings in Argentina 


103 


The Indiana University Observatory on Social Media 


"The Black and Thriving Project": The use of message 
framing and social contagion to promote vaccines in 
African American communities 


105 


Unity Consortium 


Study on Activating Vaccine Supportive Parents of Adolescents 
and Young Adults as Trusted Voices on social media 


107 


University of Antwerp 


Research to increase confidence in maternal vaccination by 
digital communication interventions 


109 


University of California, San Francisco 


WhatsApp Bots for Overcoming Vaccine Misinformation 
and Increasing Vaccine Confidence among Pregnant 
Women in Northern India: Development and Impact 
Assessment 


111 


University of Chicago 


Testing interventions to address vaccine hesitancy on 
Facebook in East and West Africa 


114 


University of Chicago, Department of Economics 


Positive reinforcement of online and offline social signaling to 
accelerate COVID vaccine Uptake 


116 


University of North Carolina at Chapel Hill 


Targeted Vaccine PSAs Based on Political Science and 
Economic Insights 


118 


University of Washington (Department of Human 
Centered Design & Engineering) 


Large-scale Studies to Advance Public's Access and Trust 
of COVID-19 Vaccine Research on social media 


120 


University of Washington (Institute for Health Metrics 
and Evaluation) 


Using social media to identify drivers of COVID-19 vaccine 
hesitancy in the US by county and ZIP code 


122 


Upstream Thinking 


Redirecting Existing Demand for Information Towards 
Vaccine Confident Interpretation by Identifying Online 
Behavioral Nudge Interventions 


124 
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ORGANIZATION NAME GEOGRAPHY 
The George Washington University USA 


David A. Broniatowski, Associate Professor of Engineering 
Management and Systems Engineering, and Lorien C. Abroms, 


Professor of Prevention and Community Health 


FOCUS POPULATION 


Unvaccinated individuals 


METHODOLOGY TYPE 
Randomized Controlled Trial 


SOCIAL MEDIA PLATFORM/S USED 


We propose to reduce vaccine hesitancy online using the microtargeting Private vaccine 
abilities and community structure of social media platforms to better under- education groups 
stand the concerns of the vaccine hesitant. We hypothesize that effective on Facebook may 
communication builds trust by addressing the gist of their concerns. Here, we help unvaccinated 
propose to deploy the GW Health Communication Volunteer Corps — a trained individuals increase 
service group for disseminating evidence-based health information to test this 


their intentions to get 


hypothesis. vaccinated. 


We hypothesized that participants randomized to the intervention group would 
develop attitudes more in favor of COVID-19 vaccination, have greater inten- 
tions to get vaccinated, and would be more likely to be vaccinated at the end 
of the study compared to the control group. Analysis of data from our trials is 
ongoing. 


Preliminary data suggests sharing information about vaccines with unvaccinat- 
ed individuals in private Facebook groups may improve vaccine confidence and 
may be sufficient to convince a small number to vaccinate, even among some 
of the most resistant individuals. Nevertheless, widespread distrust remains 
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a key barrier: personal experiences, and pandemic 
fatigue led some to continue to refuse vaccination. 

For example, we encountered several people suffering 
from long-COVID symptoms who nevertheless stood by 
their decision to remain unvaccinated and others who 
stated there was nothing that could be said to change 
their vaccination position. For people who fall into this 
category, pro-vaccine content is often viewed as an 
attempt to push a political agenda and dismissed as 
propaganda. 


Beyond increasing vaccine trust and confidence, im- 
proving vaccine uptake may rely on communicating the 
bottom-line consequences (gist) of not vaccinating in a 
manner that helps them to contextualize the decision in 
terms of their values (which must be determined through 
a sustained process of listening to their concerns). This 
is especially important when communicating to those 
who have already recovered from past infections and 
thus believe COVID to be a low-risk illness. In particular, 
we encountered many people who were misinformed 

or unaware of a variety of topics (e.g., long-COVID 
symptoms, long-term vaccination side effects, the 
inherent variability of natural immunity). Addressing 
these misconceptions led several people to vaccinate 
due to the Knowledge they gained through our program. 
Others ended the program with higher levels of trust and 
confidence in the vaccine. 


Question Baseline 
How likely are you to get a COVID-19 vaccine in the next month? Not at all likely 
Since joining this study, have you been vaccinated for COVID-19? 
How much do you trust the COVID-19 vaccine? 1—Notat all 
Do you think the COVID-19 vaccine is safe? 1 - Not at all 
Do you think the COVID-19 vaccine is effective? 2 
Overall, for you, which of the following best describes the RISKS Low 


of getting COVID-19? 


KEY PROCESS INSIGHTS 


Key insights from this pilot are that it is feasible to 
conduct vaccine education groups on Facebook; that 

it is feasible to recruit for these groups via Amazon 
Mechanical Turk, randomize participants to intervention 
and control, have participants join groups, and com- 
plete follow-up surveys with high follow up rates (e.g., 
<80%). Participants remained engaged for the duration 
of the intervention (1 month). 


Contrary to our expectations, we found that group 
members supported Facebook group moderation 
policies to limit political commentary and ban strident 
anti-vaccine comments perceived as uncivil. Most 
group members took appropriate corrective actions in 
response to moderation actions (i.e., they followed the 
community rules after being corrected), but a small 
number did not. Removing these individuals from the 
group was necessary to maintain a nonjudgmental 
space where people felt comfortable asking questions 
and sharing their viewpoints. 


Moderating large groups is a full-time job. Not every 
comment requires a response and may be counter- 
productive. As the study progressed, we settled on 

the middle-ground approach of not replying to every 
comment and creating separate discussion threads on 
commonly expressed concerns. From the moderation 
perspective, there is also the challenge of striking 

the right balance between deciding when to bring 

up vaccination and simply listening to concerns and 
expressing empathy. 


2-week 4-week 6-week 
Yes; 1 dose Yes; 1 dose Yes; 2 doses 
4 5 - Completely 
4 5 - Completely 
5 5 - Completely 
High High High 


Excerpts from free text responses 


I liked that no one was 
judgmental and that there 
was plenty of helpful 
information to help the 


people that are on the fence. 


A VACCINATION PROGRESSION, SURVEY RESPONSES 
Baseline, 2-, 4-, and 6-week survey responses demonstrating the 


progression of one subjects’ vaccination perspective. 


Through the group | found 
out that | was misinformed 
about a lot of the 
information that had me on 
the fence about the 
vaccination. 
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I was on the fence about the 
safety of the covid 
vaccination, this group 
helped me pick a side. | was 
concerned about side affects. 
The moderators showed me 
that the vaccination is safe. 


RECOMMENDATIONS 


We found that a strategy of first looking for areas 

of agreement was far more effective than trying to 
“win arguments”. As predicted, we found that we 
made more progress when we didn't “take the bait” 
of arguing numbers and verbatim details, but rather 
emphasized the gist, or bottom-line takeaway, of the 
risks of COVID and the benefits of vaccination. 


Future research may investigate a similar 
methodology with a larger number of 

participants. Additionally, future research may 
investigate message frames used in groups (e.g., 
gist vs. fact-checking) to assess the hypothesis 
that gist-based messages will outperform fact- 
based messages, group size (e.g., 50 vs. 150 
participants/group) to assess the hypothesis that 
smaller groups are more effective than larger ones, 
participant access to Facebook features (e.g., 
participants can post and comment vs. comment 
only) to assess the hypothesis that more control 
for moderators is more effective than less control, 
and composition of groups (e.g., unvaccinated 
only vs. mix of vaccinated/partially vaccinated and 
unvaccinated) to assess the hypothesis that a mix 
of vaccination statuses is more effective than a 
homogenous unvaccinated group. 
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‘W FEEDBACK ON INFORMATIVE CIVIL FORUM 


A civil forum provided participants the 
opportunity to share and receive information 
about COVID-19 vaccines. 


a 21 January © 


tam grateful for this group. | have been a part of some groups like 
this before, and this group is civil and kind. There is good 
information being shared from both sides, and good questions 
being asked! 


OO® You and 15 others 4 comments Seen by 91 


© Love © Comment 


A 

Yes, | agree & Q: 
like Reply ôv 
ae 

l agree! oO 

like Reply öv 
Completely agree! >E 


Uke Reply Ov 


Ss 

= Totally agree!!! | am super thankful to be here. Looks 
tke a great bunch! | Eke the idea of being able to share 
co-vid 19 information and get information from others 


Uke Reply 6» Q! 


‘W VALUE IN PRIVATE VACCINE EDUCATION 
GROUPS 


Participants of private vaccine education groups on 
Facebook found value in the information provided. 


lam glad they are FDA approved but | should still have 
the right decide if | want to. My body my right. 


Like Reply 


($) GW Health Communication Corps Author Admin 
es thanks for your 
comment. We agree that it should be your 
decision! Our goal is to help you get the 
information you need to make this decision. 
Please let us know if you have questions and 
we're happy to answer them! Q: 


Like Reply Commented on > [imme ) 


EEE 
GW Health Communication Corps thank 
you. | think this forum has been very 
informative! 
like Reply j © - 


GEOGRAPHY 
Canada, USA 


ORGANIZATION NAME 
19 to Zero Inc. 


FOCUS POPULATION 
Black, Indigenous, Latinx, and LEBTQ+ communities 


METHODOLOGY TYPE 


Supervised machine learning to identify marginalized community 
Twitter accounts and Tweets; unsupervised clustering model and 
topic analysis. 


SOCIAL MEDIA PLATFORM/S USED 


vy 


Understanding and tackling social media sentiment is critical to increasing vaccine 
uptake, particularly in vulnerable populations. In conjunction with community partners, 
we will use Natural Language Processing algorithms to systematically analyze senti- 
ment in specific vulnerable populations and work to develop effective messaging to 
promote vaccination. 


From the supervised machine learning model, we identified 2156 Black, 5198 
Indigenous, 10123 Latinx, 14454 LGBTQ+ Twitter accounts and 10958 Black, 17674 
Indigenous, 38476 Latinx, and 56195 LGBTQ+ tweets. First, among Black, Indigenous, 
and Latinx groups, mistrust in institutions and elites was the prominent theme in 

the vaccine hesitant conversations, which may be rooted in historical mistrust and 
systemic racism. Second, the topic analysis showed that the LGBTQ+ community also 
expressed concerns for vaccine safety and mistrust in institutions within the vaccine 
hesitant discussions. Third, contrary to vaccine hesitant individuals, vaccine confident 
members within marginalized communities were predominantly concerned about 
vaccine access, reflecting inequities in vaccine distribution. 


Overall, machine learning can be used to identify attitudes towards COVID-19 vaccines 
among marginalized populations on Twitter. By leveraging the computational power 
of machine learning, large volumes of social media datasets can be analyzed to 
inform public health decision making, empower policy makers, and enhance vaccine 
confidence among marginalized groups. 


Our research 
used machine 
learning 

to identify 
themes 
surrounding 
COVID-19 
vaccines 

on social 
media among 
marginalized 
communities, 
which can 

be used to 
enhance 
vaccine 
confidence and 
access. 
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KEY PROCESS INSIGHTS 


To effectively use machine learning insights to guide 
decision making, one key principle is the concept of 
algorithm literacy. Often, artificial intelligence can be 
perceived as a “black box” by stakeholders unfamiliar 
with the technology. Although decision makers do not 
need to be experts in data analytics, insights should 
nevertheless be communicated in a way that is easily 
understood. Suggestions include using diagrams to 
highlight the inputs and outputs of the machine learning 
model, graphs to visualize the data, and annotations 

to highlight key insights behind the analysis. As the 

use of artificial intelligence continues to expand in the 
world of public health, it is integral that technical jargon 
is minimized to improve the effectiveness of Al in 
improving population health. 


Due to the shifting nature of public health guidelines, 
community members have often expressed fatigue 

when it came to discussing COVID-19. However, we took 
measures to engage them in ways that emphasize the 
unique methodology of this project (i.e., Al-generated 
insights) and how it has wider implications for other 
health challenges. When it came to the principles of 
vaccine confidence messaging, we recognized that simply 
providing information was insufficient for behavioral 
change. Instead, actively listening to the voices of 
community members was key to building understanding 
and addressing their concerns. Furthermore, this feedback 


SENTIMENT ANALYSIS VACCINE CONFIDENCE 
Sentiments around mistrust, access, and efficacy. 
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from community members has allowed us to shift the 
way we approach our community engagement efforts: 
focusing on topics they care about first, considering their 
broader social and psychosocial supports and meeting 
them where they are. 


RECOMMENDATIONS 


Advocate for open-access data from social media 
companies (e.g., Twitter, Facebook) to enhance 

volume of inputs for machine learning. This can help 
improve accuracy of insights generated from natural 
language processing models. Increasing access (e.g., 
by reducing costs) for supplementary data points 

such as geolocation data for public health / prosocial 
initiatives can also help advance research efforts aimed 
at addressing inequities and improving health. 


Establish and promote a Community of Practice among 
a wide set of stakeholders (governments, NGOs, 
healthcare providers) to support community partners 
in sharing/developing/tailoring materials for engaging 
with their members. Often, the rapidly changing 
nature of COVID-19, vaccines and therapeutics can 
be challenging for any one organization to overcome 
on their own. While there have been pockets of 
collaboration across the board, continuing to break 
down silos and enhance networks of support will be 
essential to reducing the burden that all stakeholders 
feel in building vaccine confidence. 


Topic Themes 
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ORGANIZATION NAME GEOGRAPHY 
Busara Center for Behavioural Economics Kenya (primary), Nepal, Philippines 


Kelly Zhang, Busara Center for Behavioral Economics and MIT 
Governance Lab 


Chaning Jang, Busara Center for Behavioral Economics 
Nicholas Otis, UC Berkeley 

Lily Tsai, MIT Governance Lab 

Alisa Zomer, MIT Governance Lab 


FOCUS POPULATION 
Social media users 


SOCIAL MEDIA PLATFORM/S USED 


METHODOLOGY TYPE f 


Crowdsourcing videos and surveys; online news feed experiment 
and surveys 


We use a novel approach in crowdsourcing original content from social media users 
in Kenya, Nepal, and the Philippines to build and test locally-driven COVID-19 vaccine 
campaigns. First, it crowdsources social media content for different types of vaccine 
messaging. Second, it uses survey experiments to test which types of messaging 
are the most effective across country contexts. Third, it uses news feed experiments 
to evaluate the extent to which top messaging strategies are effective within a com- 
petitive information environment for each country. We use survey measures to track 
vaccine confidence and behavioral measures to track vaccine uptake. 


The study had two main research objectives: 1) Feasibility of crowdsourcing 
COVID-19 video content from social media users; 2) Test if local vs. international 
COVID-19 video content is more effective across contexts to increase vaccine 
willingness and misinformation. 


Crowdsourcing 
local science- 
based video 
content to 
address 
vaccine 
hesitancy and 
misinformation 
is challenging 
but possible. 
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The first objective was tested rigorously in Kenya, our 
primary country context, with the following high-level 
results: 


e Social media users watch videos and can answer 
questions on the content (a proxy measure for if 
people watch videos and pay attention). In a survey 
of 12,000 users, a majority of video watchers (90%) 


correctly answered a content question on the video. 


e Itis possible to run targeted campaigns on social 
media where you get random people to create a 
video focused on a specific issue. Of 4,400 social 
media users who submitted a test video, around 
~450 people uploaded a persuasive video to help 
get people vaccinated. 


e Crowdsourced video content improves when 
instructional infographics are provided about 
how to film a good video, including tips on lighting, 
sound, framing, and eye-contact. 


MAKE SURE YOUR 
SURROUNDINGS ARE 
WELL LIT. 


2. AUDIO: 


A survey of people who produce videos suggests 
that this group have strong beliefs about what 
messaging they think will be persuasive. For 
instance, they think positive videos are more 
convincing (75%) than negative videos, and that 
videos of women are better at convincing men and 
that men are better at convincing women. 


Perhaps unsurprisingly, people who submit videos 
come up with creative and intuitive strategies to 
persuade people that are consistent with behavior- 
al theory. For example, highlighting social norms, 
downplaying risks, emphasizing social benefits and 
economic effects, peace of mind and stress reduc- 
tion were all strategies that were used. Storytelling 
(e.g., people who were affected by COVID-19) was 
also used as a common motivator. 


CHECK THAT YOUR HEAD IS 
FULLY IN THE FRAME, AND YOU'RE 
HOLDING THE PHONE A BIT 
HIGHER THAN YOUR EYE LEVEL. 


TAKE THE VIDEO IN A PLACE HOLD THE CAMERA AS STILL 

WHICH DOESNT HAVE TOO ~ AS YOU CAN, AND TRY TO 

MUCH BACKGROUND NOISE. AVOID WALKING AROUND. 

ALSO ENSURE THAT YOU'RE 

NOT TALKING TOO QUIETLY. 
ENSURING 
HIGH- 
QUALITY 
VIDEOS 

TIT Providing 
MAINTAIN EYE CONTACT WITH SHOOT YOUR VIDEO MULTIPLE TIMES IF infog al 
THE CAMERA, OR AT LEAST LOOK YOU NEED TO. MAKING SMALL on capturing 
a ADJUSTMENTS TO ANGLES, LIGHTING, 4 

IN THE DIRECTION OF IT. YOUR WORTS! AND SOJON THIS WAY. videos helped 
to ensure 
high-quality 
results. 
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Sourcing local content videos across contexts: 


e Replicating a smaller version of an online contest 
to source local content videos across Kenya, 
Nepal and the Philippines proved more challeng- 
ing. For example, spending the same in advertising 
across a Set period (2-3 weeks) produced a few 
videos in Kenya and the Philippines, but no results 
in Nepal. Doubling the contest reward in Nepal also 
failed to produce more video content. As a result, 
we had to send individual messages to get locally 
produced videos in Nepal, as opposed to randomly 
through social media advertising. More on this in 
key process insights below. 


For the second research objective on the effectiveness 
of local content against international content from the 
World Health Organization (WHO), we include below 
initial pilot baseline measures from Kenya. These 
measures are based on a small sample size and may 
change with additional data collection. 


e With the current pilot sample, there is no treatment 
effect, so we can't compare across videos. We 
will need to complete the full study to see if these 
results hold or if the sample size is too small. 


e We do have some findings about specific groups 
and individual videos. For example, initial results 
suggest that WHO content can be most effective. 
1) Individuals who trust science are more likely 
to pledge to get a COVID-19 vaccine after seeing 
WHO content; 2) Individuals who know where to 
get a vaccine are more likely to agree with state- 
ments that they would go and get vaccinated for 


their friends and family after seeing WHO content. 


Additional follow ups and analysis are underway to 
further unpack these initial results, and we are also 
gaining input from stakeholder organizations to 
interpret these findings. 


KEY PROCESS INSIGHTS 


Below we include key process lessons learned on the 
technical and the substantive side of running online 


video contests and surveys in our selected geographies: 


Invest in the technical infrastructure necessary to 
run complex campaigns and multi-stage surveys 
in developing country contexts. The rationale for 
our target geographies was the paucity of rigorous 
online experiments in these contexts. As a result, 
we also had to build the online campaigns and 
experiments infrastructure from scratch in parallel 
with implementing an ambitious multi-country 
study. A key hurdle was figuring how to process 
thousands of small survey payments and incen- 
tives across geographies with different policies in 

a cost-effective way. Investment in infrastructure 
will continue to serve and expand for future studies, 
but it was a big challenge to build out, and required 
dedicated engineering, technical resources, time, 
and changes to our study protocol. 


Balancing the timeliness of crowd-sourced treat- 
ments with research timelines and processes. The 
benefit of crowd-sourced content is its relevance 
and timeliness, but it took time between getting the 
videos and implementing the surveys. This reality, 
combined with the rapidly changing COVID-19 infor- 
mation ecosystem, made us question if the content 
was still appropriate and relevant (for example, the 
WHO treatment video on “herd immunity” may have 
lost persuasiveness over time). Our process here 
may have been too methodical or slow to capture 
the spirit and usefulness of crowdsourced content. 


Adapting across contexts took experimentation, 
time, and did not always yield results. Though 
social media usage numbers are comparable across 
Kenya and Nepal, with the Philippines leading in 
numbers, in practice it was much more challenging 
to replicate our process to obtain crowdsourced 
videos and build survey pools in each country. For 
example, despite contextualizing online advertise- 
ments in local languages, the same amount of 
money spent on ads in Kenya and the Philippines 
resulted in very little uptake in Nepal. Even doubling 
the compensation for video submissions did not 
impact engagement numbers. These contextual 
differences were important learnings and required 
iterative experimentation and adaptation of the study 
protocol. They also served as a justification for the 
country selection and replication, to build knowledge 
in and across these understudied contexts. 
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RECOMMENDATIONS 


We are currently in the process of getting feedback 
from various stakeholders and partners in-country 


on our initial results to further refine specific 
recommendations and messaging for health 
implementers. These early recommendations are 
preliminary and may undergo additional iterations. 
We welcome input or suggestions on their utility and 
applicability across contexts. 


High-level: 


To gather better insights into attitudes and 
behavior and to close the research gap, we need 
to invest heavily in online research infrastructure. 
Online social media and information campaigns 
will continue to be an increasingly important 
medium for public health outreach. To gain 
insights on effective public health messaging and 
compliance with health measures and policies, we 
need to invest in online infrastructure to better test 
messaging and behavioral outcomes. Technically, 
this requires better integration across social media, 
pixel tracking, survey platforms (e.g., Facebook 
and Qualtrics), and cost-efficient infrastructure for 
incentives and payments to respondents. 


Sourcing local content is possible but challeng- 
ing, and varies significantly by context. To further 
test the utility of crowdsourcing requires resources, 
time, and patience for experimentation and 
adaptation. We had to play around with ad spend, 
incentives, and treatment instructions before 
seeing outputs in terms of videos and surveys. 
This exploratory process was repeated in each 
country’s context to varying results. A comparable 
small-scale campaign run across countries was 
more successful in Kenya and the Philippines, and 
less so in Nepal. We were limited here because we 
could not find a cost-effective way to make survey 
payment for incentives across countries. 
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Substantive: 


Build capacity to enable rapid testing for 

public health campaigns on social media with 
timeliness and relevance as priorities. Social 
media and information ecosystems change rapidly, 
especially in the context of COVID-19 variants and 
pandemic developments, requiring flexibility and 
creativity. What we thought would be relevant and 
useful in the proposal stage changed significantly 
throughout the study period as international and 
local movements and variants led to rapid policy 
change around mask-wearing, lockdowns, curfews 
and travel, and resources (as well as attitudes) 

for vaccination. Crowdsourced content, which is 
timely and perhaps short-lived by nature, requires a 
shorter test horizon. 


In Kenya, social media users watch and pay 
attention to videos online; they have strong beliefs 
of what makes for persuasive content. In our 
primary study context of Kenya, a survey of people 
who produce videos for Facebook felt strongly that 
video content can persuade, that positive videos are 
more persuasive than negative ones, and that people 
of the opposite gender are more convincing messen- 
gers (i.¢., women are better at convincing men). 


To effectively target content-specific messages, 
we need to understand people's baseline beliefs. 
Initial baseline pilot results show that some video 
campaigns might be more effective for certain 
subsets of the populations. We need to know more 
about people's baseline beliefs to effectively target 
content-specific messaging, requiring additional 
infrastructure and data. 


GEOGRAPHY 


USA and Cameroon 


ORGANIZATION NAME 
Center for Media Engagement at the University of Texas at Austin 
Lucy Atkinson, The University of Texas at Austin 


Sean Upshaw, The University of Texas at Austin 


Lillie D. Williamson, University of Wisconsin-Madison 
Jean-Louis Ntang Beb, Yaoundé University 

Lee Ann Kahlor, The University of Texas at Austin 
Kristen Sussman", The University of Texas at Austin 
Josh Anderson, The University of Texas at Austin 


KEY PARTNERS 
Viamo and YouGov 


SOCIAL MEDIA PLATFORM/S USED 


FOCUS POPULATION 
Black Americans and Cameroonians 


METHODOLOGY TYPE 
Mixed methods: Survey and quasi-experimental 


For public-health campaigns to drive demonstrable reductions in vaccine hesitancy, 
messages must do more than fill a lack of knowledge - they must focus on the audience's 
deeply held beliefs. Our project addresses vaccine hesitancy among Black Americans 
and Black Africans. We assert that medical mistrust, doubts about vaccine efficacy, 

and perceived racism in health care are major sources of hesitancy. Informed by Moral 
Foundations Theory, we will collect panel survey data in the U.S. and Cameroon to 
explore vaccine hesitancy and medical mistrust. We will then develop and deliver morally 
resonant, historically sensitive social media interventions to reduce COVID-19 vaccine 
hesitancy and bolster vaccine uptake. 


Our preliminary results suggest three key findings for health communicators. First, COVID-19 
is less of a concern in Cameroon. According to our survey results that were conducted in April 
2022, about one in 20 (about 5%) Cameroonians believed they have had COVID-19, whereas 
one in five Black Americans (about 20%) believes that they've had COVID-19. Our sense is that 
this goes beyond the general lack of vaccine availability and relates to more deeply rooted 
between-country differences in culture and COVID-19 epidemiology. 


Second, our survey results show that vaccine hesitancy and race-based medical mistrust 
vary between Black Americans and Cameroonians. Further, moral foundations theory 
subscales were inconsistent in reliability in the U.S. Black American samples. Specifically, 
in the U.S. the care/harm, fairness/cheating, purity/degradation foundations maintained a 
good reliability standard above 70%, but loyalty/betrayal (a = .55) and authority/subversion 
(a = .67) foundations did not. For the Cameroonian samples, however, the foundation 


For public-health 
campaigns to 
be effective 

in driving 
demonstratable 
reductions 

in vaccine 
hesitancy 
among Black 
people, 

social media 
messaging 
should be 
framed in a way 
that addresses 
variations in 
moral values, 
culture, and 
communities. 
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subscales resulted in low reliability on all subscales, 
which ranged in alpha between 29% and 47%. This 
suggests that morally framed appeals may not work as 
intended in Cameroon. 


Third, social media communicators should test artificial 
intelligence (Al). Our quasi-experimental study found 
that messaging deployed on Facebook that has been 
enhanced using Al can outperform human-generated 
communication. The results from our pilot study found 
that there was a statistically significantly difference 

in impact of click-through-rate in the Al-mediated 
communication (AI-MC), which means that as more 
people were shown the ads with AI-MC, more people 
behaved with our goal in mind to click through and 
“learn more.” 


KEY PROCESS INSIGHTS 


Our work uncovered insights related to key challenges 
of conducting our research in Africa. Some of these 
challenges were highlighted by our attempt to conduct 
and compare survey collections in both the U.S. and 
Cameroon. Due to low mobile network coverage and 
low infrastructure development with telecommunica- 
tions in Cameroon, the internet remains very expensive 
and limited, so in-person interviews were required for 
data collection. Interview fieldwork in Cameroon had to 
overcome more difficulties related to training, research 
delays, and travel restrictions due to safety concerns. 
The geographical characteristics led to long-distance 
travel for the data collection team to reach the selected 
communes in the regions. Additionally, with a restricted 
mode of transport, interview teams often had to wait 
for a specific day of the week to travel to a particular 


(Padamse et al., 2022). 
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Medical mistrust, doubts about COVID-19 vaccine efficacy, and 
perceived racism in healthcare are the source of vaccine hesitancy 
(Bogart et al., 2021; Dinga, Sinda, & Titanji, 2021; Seydou, 2021). 


Black Americans are especially accessible on social media, and use it in 
higher numbers than white Americans, with 74% of Black Americans on 
Facebook and 49% on Instagram (Auxier & Anderson, 2021). 


community. Two days of traveling was required most of 
the time to reach a locality. Security also was a concern, 
particularly in the northern portion of the nation, where 
insurgents are present. As a result, a state of emergen- 
cy has been declared in the areas, prohibiting the data 
team from leaving at a specific hour of the day. For 
example, two nights are required before reaching the 
destination when traveling inside these localities, since 
night travel by bus or motorcycle is not safe. These 

are important realities when conducting comparative 
(between-country) research. 


RECOMMENDATIONS 


Social media allows for detailed targeting, but 
interpreting and translating these insights into the 
targeting is crucial. Specifically, what are the beliefs and 
experiences that contribute to Black people's psycho- 
logical attributes, how do they vary across nations and 
communities, and what messaging will best resonate 
to encourage vaccine uptake? Vaccination rates in 
Black Americans are on the rise and are accelerating 
faster when compared to White Americans. However, 

in Cameroon, fewer people think they've had it, fewer 
people are vaccinated against it, and fewer people plan 
to get vaccinated. Health communicators have a bigger 
challenge in accelerating vaccine rates in Cameroon. 
Interestingly, of the moral foundation questions used in 
our survey, "vulnerability of others" seemed to perform 
similarly across CM/US suggesting that Cameroonians 
do express similar concern when people's weaknesses 
are taken advantage of. Communicators could consider 
using a moral foundations of care framework in vaccine 
messaging that address our vulnerability to injury or 
inevitable dependencies (Miller, 2020). 


BACKGROUND 
ON COVID-19 
VACCINE 
RATES 


Key factors 
for vaccine 
hesitency 
in Black 
Americans. 


Vaccine rates in Black Americans have accelerated suggesting that 
vaccine hesitancy is malleable, at least in the context of COVID-19 


ORGANIZATION NAME GEOGRAPHY 
Corowa-kun Japan 
Leading researchers: 

Takaaki Kobayashi MD 


Division of Infectious Disease, Department of Internal Medicine, 
University of lowa Hospitals & Clinics, Ilowa City, IA, USA 


Hana Tomoi 


MSc, PhD candidate, Faculty of Epidemiology and Population 
Health, London School of Hygiene and Tropical Medicine, Keppel 
Street, London, UK 


Yuji Yamada MD 


Brookdale Department of Geriatrics and Palliative Medicine, Icahn 


School of Medicine at Mount Sinai, New York, New York, USA MEN ORION AE 


3-arm randomized trial (control group vs. chatbot group vs. 


webinar group) 
KEY PARTNERS 


The Laboratory of Data Discovery for Health Limited, The 
University of Hong Kong, China. 


FOCUS POPULATION 


Japanese persons who did not receive any COVID-19 vaccine and 
did not intend to receive it in the future 


We conducted a three-arm randomized controlled trial. The study period was between 

5 November 2021 and 9 January 2022. Japanese who were aged = 20 and who had not 
received any COVID-19 vaccine and did not intend to be vaccinated against COVID-19 in 
the future were recruited and randomly assigned to one of the following three groups: 

(i) a control group (with no intervention), (ii) a group with a free chatbot in a popular 
messenger app called ‘LINE’, which provided general information on COVID-19 vaccines 
and (iii) a group with free webinars where health professionals interactively provided 
participants with the information on COVID-19 vaccines. The format of each webinar was 
the combination with a 10-minute presentation and a 20-minute live Q and A (up to a total 
of 60 minutes if 30 minutes were not enough to answer all questions). 


We performed pre- and post-intervention cross-sectional surveys on all three groups to 

compare the change in the vaccine intention and Vaccine Confidence Index (VCI) before 
and after interventions between the control group vs. the chatbot group and the control 

group vs. the webinar group. 


A total of 1,158 persons who had not received COVID-19 vaccines and not intended 

to be vaccinated in the future were randomly assigned to three different groups: 386 
participants in the control group, 386 in the chatbot, and 386 in the webinar group. The 
baseline characteristics and demographics were balanced across three groups for most 
variables measured. 


SOCIAL MEDIA PLATFORM/S USED 
ga Messenger app “LINE” for chatbot 


Si “Zoom” for interactive webinars 


Interactive 
webinars with 
live Q and 

A provided 
by health 
professionals 
could have 

a role in 
increasing 
COVID-19 
vaccine 
confidence. 
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Among 386 persons assigned to the chatbot group, 
237(61.4%) accessed and used the chatbot at least 
once during the study period. Of them, 231 (97.5%) 
answered the post-survey.359/386 (93.0%) in the 
control group answered the post-survey. There was 
no difference in the vaccine intention (the willingness 
to be vaccinated against COVID-19) in the post-survey 
between the chatbot and the control group (14.7% in 
the chatbot group and 18.7% in the control group). 
There was also no difference in VCI for the importance, 
safety, or effectiveness between the two groups. 


Among 386 persons assigned to the webinar group, 
215(55.7%) attended the webinars at least once. Of 
them, 207 (96.3%) answered the post-survey and 
compared to the control group. There was no difference 
in the vaccine intention in the post-survey between 

the webinar group and control group. (18.8% in the 
webinar group and 18.7% in the control group). The VCI 
for the importance and the effectiveness significantly 
increased in the webinar group compared to the control 
group. However, there was no difference in VCI for the 
safety between the two groups. 


VCI for the importance and the effectiveness in the 
control group decreased without any intervention 
during the study period. 


KEY PROCESS INSIGHTS 


Our randomized control trial demonstrated that neither 
the chatbot nor the webinar changed the vaccine 
intention (the willingness to be vaccinated against 
COVID-19) among those with COVID-19 vaccine 
hesitancy in Japan. However, VCI for the importance 
and the effectiveness significantly increased with the 
webinar intervention, whereas VCI for the safety did 
not change. The chatbot did not increase VCI. On the 
other hand, the more frequent use of the chatbot was 
associated with increased vaccine intention. In addition, 
VCI for the importance and effectiveness naturally 
decreased in the control group during the study period. 
It might suggest that the degree of vaccine hesitancy 
might worsen as time goes on in those who did not 
intend to be vaccinated against COVID-19. 


Interactive webinars with live Q and A provided by 
professionals could have a role in increasing COVID-19 
vaccine confidence. 
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RECOMMENDATIONS 


Our study revealed that webinars increased the 

Vaccine Confidence Index (VCI) for the importance 

and effectiveness of the COVID-19 vaccine. There are 
currently many webinars available for COVID-19 general 
information and vaccinations worldwide. However, 
most webinars are just presentations by professionals 
and have no opportunities for the audience to ask 
questions and get answers in a timely manner (mostly, 
webinars are just one-way communication [presenters 
-> audience]). We hypothesize that the interactive and 
relatively small-group webinar (with live Q and A be- 
tween professionals and audience during the webinar) 
might effectively increase vaccine confidence. Local 
public health agencies and healthcare associations 
should consider more interactive webinars using local 
professionals, especially in areas with the low COVID-19 
vaccine uptake. 


VACCINE INTENTION AND CONFIDENCE AFTER 
INTERVENTIONS 


All Group 1 | Group2 | Group3 Group Group 
participants | (control) | (chatbot) | (webinar) | 1 vs 2, 1vs 3, 
n=797 n=359 n=231 n=207 P-value? | P-value? 
Willing to be 18.7 14.7 18.8 0.259 1.000 
vaccinated 

b, % 

Change in vaccine confidence € 

COVID-19 -2.2 -0.8 8:7 0.729 0.009 
vaccines are 

important, % 

COVID-19 1.9 1.8 4.8 0.865 0.303 
vaccines are 

safe, % 

COVID-19 -8.1 -3.4 5.3 0.249 0.002 


vaccines are 
effective, % 


a Difference in willingness to be vaccinated was assessed by Chi-square test; 
changes in vaccine confidence index were assessed by mixed effects logistic 
regression model. 


> Including those who have received COVID-19 vaccine or have not taken 
COVID-19 vaccine but are willing to receive 


c difference in % (post-intervention — pre-intervention) of those who have 
responded “strongly agree” or “tend to agree.” (Other responses are “do not know,’ 
“tend to disagree,” “strongly disagree”) 


Though a chatbot did not change the vaccine intention or VCI in our study, the more frequent use of chatbot was 
associated with increased vaccine intention. If we improve the contents and format of our chatbot so people 

will be more likely to access and use it, it could significantly impact vaccine hesitancy. For example, our current 
chatbot is not supported by the natural language process where people can enter free text; currently chatbot users 
can access pre-defined Q and A only. Therefore, some users might not be able to find the individualized answers 
that they want to know for their specific situation or status. However, given that a chatbot can be relatively easily 
created, local agencies can consider implementing it to disseminate accurate COVID-19 vaccines information. The 
chatbot has the possibility of increasing vaccine intention depending on its accessibility and usability. 


386 Randomized to no 


Allocation intervention 


Intervention 


359 Answered a post- 


Post-survey survey 


STUDY FLOW DIAGRAM 


99,965 Assessed for eligibility 


1,158 Answered a pre- 
survey and randomized 


98,807 Excluded 
- Received COVID-19 vaccines (n=84,567) 


- Planned to receive vaccines (n=2,084) 

- Had used the chatbot already (n=144) 

- Declined to use the chatbot or attend online 
webinars (n=12,012) 


386 Randomized to the 386 Randomized to online 


chatbot 


webinars 


237 Used the chatbot 215 Attended online 


webinars 


231 Answered a post- 207 Answered a post- 


survey 


survey 


| 


Analysis 359 Analyzed 231 Analyzed 207 Analyzed 


Methodology and process showing the number of participants 
engaged in Corowan-kun Supporters’ project. 
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ORGANIZATION NAME 


Grameen Foundation India Pvt Limited GEOGRAPHY 


India 
FOCUS POPULATION 


Rural low-income households in India 


METHODOLOGY TYPE 


Quasi-experimental 


SOCIAL MEDIA PLATFORM/S USED 


ie e 


The objective of the project is to assess the role of social media in resolving information asymmetry 
and driving COVID-19 vaccination confidence & uptake amongst low-income households in rural 
India. The experiments will compare and contrast the efficacy of social media channels in resolving 
vaccination hesitancy and driving vaccine uptake through phygital model and digital model. Focus 
areas for the intervention are Uttar Pradesh, Maharashtra, Haryana & Tamil Nadu in India. 


As aresult of these experiments, the vaccine hesitancy (7.6%) and information asymmetry (6%) 
amongst the rural low-income households in the intervention area was reduced and significant 
improvement in vaccination confidence and uptake was reported. The program increased confidence 
in vaccines among 7.6% of beneficiaries and filled the gap in accessible and accurate information 

on COVID-19 vaccines for 5.7% of beneficiaries. There was an incremental increase (7%) in the trust 
level on social media channels for COVID-19 related information amongst the program beneficiaries. 
The comparison of the impact across the digital and phygital arms revealed that the digital model 
was more efficient and impactful than the phygital model in resolving information asymmetry, 
vaccine hesitancy, and vaccination confidence. The digital model was able to reduce a greater 
proportion of vaccine hesitant persons (by 17%) than the phygital beneficiaries (16%). Similarly, the 
incremental increase in vaccine uptake in the digital model was by 2%. Moreover, the digital model 
required fewer resources to implement since it eliminated physical contact requirements and was 
more convenient and scalable. However, low penetration of internet and internet-enabled devices is a 
challenge that is being addressed by the government's telecom policies at a steadily fast pace. 
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Social media 
experiment 
leads to 
7.6% lesser 
vaccine 
hesitancy 
and 1.7% 
higher 
vaccine 
adoption in 
rural India. 


KEY PROCESS INSIGHTS 


T: 


The development of the educational videos was 
informed by the baseline survey, FGDs, and the 
on-field interactions of implementing partners with 
the intended beneficiaries of the program. The need 
for curated information was identified, taking into 
account all the myths and misconceptions that were 
floating around in the local communities, and videos 
were crafted to address all of these issues by making 
use of Cialdini’s principles of influence. For example, 
amidst the complex network of myths about potential 
deadly side-effects of COVID-19 vaccinations, most 
lactating mothers and pregnant women decided 

not to get the vaccine, as it might affect the children 
through the breast milk or inside the womb. This 
resulted in significantly lower vaccination rates, 
specifically for women with children. So, a video 

to educate these women about the benefits of the 
vaccine was prepared, leveraging the social proof and 
liking principles of Cialdini. Hence, generating insights 
and evidence from the ground, and then developing 
the educational content, has proved a dependable 
strategy for the program. 


Engagement of local community persons and influ- 
encers has proved to be another big strength of the 
D-VACSI program. The on-boarding of beneficiaries 
and building of their trust in the program was achieved 
through engaging local influencers like Grameen 
Mittras, ASHA/ ANM, frontline health workers, and 
local leaders in reaching out to the beneficiaries. 


The digital content (videos) for awareness generation 
under the D-VACSI program were created to give 

the beneficiaries exposure to certain key messages. 
These key messages were designed and drafted using 
Cialdini's principles of persuasion. It was observed 
that a lot of people were motivated by the content 
and, hence, efficacy of Cialdini’s principle in social 
interventions was established. 


In spite of COVID-19's second wave, beneficiaries 
attended the training, as trainings were imparted 
digitally/online where beneficiaries could attend 
trainings from their respective homes and adhere to 
COVID-19 protocols. 


Physical meetings with government officials and 
frontline workers facilitated in getting the list of 
non-vaccinated and vaccinated persons, and they 
supported in the execution of the project. 


RECOMMENDATIONS 
iE 


vy 


Distrust in various 


Vaccine stakeholders (change in % of 


Informatio 


Rural people have very high faith in local health 
workers, especially ASHA workers. Implementing 
any health intervention through them increases the 
likelihood of success of the intervention significantly. 


Cialdini’s principles of influence have been found to be 
effective in increasing people's faith in vaccines and 
decreasing their vaccine hesitancy. Any key messages 
intended to reach masses may leverage these prin- 
ciples, especially messages which have elements of 
social proofing, authority, and liking. 


Pregnant women and lactating mothers are found 
to have the highest level of vaccine hesitancy. Any 
intervention keeping them at focus may increase the 
overall impact of the program significantly. 


Results from the study also provided insights for the 
curation and development of messages for driving 
health seeking behavior, especially messaging. High 
uptake and retention of messages, following the 
principles of social proofing, liking, and authority, 

can inform future programming and communication 
strategies that target health-seeking behavior changes 
for low-income households. 


D-VACSI PROGRAM IMPACT 
Changes in respondents confidence and trust. 


Change in % of respondents having trust in social... ES 6.8 


Change in % of respondents having enough... 57 


n 


M44 


Change in % of respondents having enough... 
Change in % of respondents ready to get their... «5.6 
Change in % of respondents ready to get vaccinated 76 


Change in % of respondent thinking vaccine can... -63 = 


yaccine confidence asymmetry 


Change in % of respondent thinking vaccine can...-7.0 = 


Social media 5.6 ME 
Local leaders 55 M 
roj ASHA/ANM/AWW -2.1 E 
Health workers, PHCs/ CHCs/ District hospitals 55 M 
Elected Leader such as PM/CM/MP/MLA Ma 3.2 


Change in vaccine hesitant population (%) 


hesitancy respondents having distrust Distrustin Vacdne 


and uptake 


Change in vaccine uptake (%) 


VCF PRELIMINARY IMPACT REPORT | | 63 


ORGANIZATION NAME GEOGRAPHY 


Institute for Global Health Sciences, Guatemala 
University of California, San Francisco 


KEY PARTNERS 


Wuqu’ Kawoq| Maya Health Alliance 


Digital Medic, Stanford University 
Upswell 


FOCUS POPULATION 


Guatemalan Indigenous communities 
SOCIAL MEDIA PLATFORM/S USED 


METHODOLOGY TYPE p ENET E 


Formative qualitative 
Brand Lift 


Pre/post surveys in communities 


Our team, comprised of public health researchers and epidemiologists, digital behavior A culturally and 
change experts, and an organization working to improve the health of indigenous com- linguistically 
munities in rural Guatemala, aims to refine already ongoing messaging about vaccine concordant 
hesitancy by combining insight from communities with expertise gained from previous community 
social media messaging campaigns. We will then roll out our social media campaign and informed 
test the impact on vaccine uptake at local health centers, as well as on vaccine hesitancy campaign using 
and other beliefs. social media 
increased 
vaccination 


Brief, culturally and linguistically tailored videos addressing COVID-19 vaccine misinforma- rates in a 

ion and knowledge deployed over social media may improve vaccine rates in rural, hard to predominantly 
reach, indigenous populations in Guatemala. To test this hypothesis, our team conducted indigenous 
formative research, developed a social media campaign, and tested it in two ways: via 
Brand Lift and via in-person pre/post surveys. Our formative qualitative interviews aimed 
o understand common barriers to vaccination and found a general dearth of information 
for indigenous communities, mis- and dis-information, general fears, and access and 
supply barriers. Following this, we designed informational videos in local languages 

o deploy over social media. To evaluate these, we first conducted a Brand Lift study 
which showed that videos in Mayan languages appear to improve the perceived safety 
of COVID-19 vaccines, while Spanish videos may have improved perceived acceptability 
of COVID-19 vaccines for Spanish speakers. Finally, we conducted an evaluation using 


population in 
rural Guatemala. 
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in-person surveys with members of indigenous com- 
munities before and after deploying the social media 
video campaign and found increased odds (1.83 times 
the odds) of getting vaccinated among those who saw 
our videos over social media compared to those who 
did not, adjusted by age, community, sex and language 
spoken at home. Findings from this study can help key 
stakeholders at the national and local level consider 
optimal practices for development of vaccine education 
content and how to leverage this content over social 
media to increase COVID-19 vaccine uptake in other 
regions of the country. 


KEY PROCESS INSIGHTS 


e Working with a community organization deeply 
trusted by the community who has significant 
experience in conducting research allowed us to 
design our research methods around the needs of 
the community. 


e |t also facilitated rapid implementation of research 
on the ground once approvals had been received. 


e In-person surveys followed by telephone surveys to 
measure the on the ground impact of our inter- 
vention allowed us to minimize in-person contact 
thus decreasing risk of COVID-19 transmission to 
research staff and participants. 


e We aimed to strike a balance between creating 
animated content that could be quickly and cost-ef- 
fectively adapted for multiple contexts through 
minimal visual edits, but also contextualized to 
specific community needs through translation and 
local language voice over. 


e Applying the principles of human centered design 
allowed us to successfully gather nuanced feed- 
back from members of indigenous communities to 
develop content that directly addressed concerns 
and misinformation circulating in communities. 


e We had methodologic challenges with the Brand 
Lift study given the small population of Maya 
language speakers in Guatemala and the fact that 
these languages do not exist on Facebook. 


e A limitation of the Brand Lift tool is that two studies 
cannot run in the same geographic regions without 
overlap. To avoid contamination (i.e., users being 
shown ads from two different Brand Lift studies), 
we geographically separated the two studies based 
on region. 


RECOMMENDATIONS 


1. Community based organizations (CBOs) have 
pre-existing priorities and projects that may 
compete for time and human resources. Thus, any 
project under development must be designed to 
ensure the research, and potential implementation 
project developed from that research, serves the 
organization's purpose and mission from the 
beginning. 


2. Research-informed interventions can increase 
activity effectiveness. Research can help guide 
CBOs towards more efficient and effective targeting 
of interventions. 


3. Trust is critical. Numerous interviews and focus 
groups indicated that local healthcare workers, 
including community health workers and local 
councils, were trusted sources of information. 
Understanding local dynamics around who is 
trusted and why can allow for better targeting of 
resources. Trusted community organizations and 
leaders can likely have an outsized impact on lives 
when trust in central government is low. 


4. Technology and social media can be leveraged to 


change behaviors. As social media use increases 
in historically marginalized communities, it can be 
used as a platform for community organizations 
and leaders to communicate and influence health 
decision making. 


5. Community voices MUST be front and central 
in dissemination activities. Ensuring community 
partners are leading the way on podcasts, inter- 
views with press, and are active members and 
authors on journal article writing teams is critical 
for appropriate recognition, capacity building, and 
continuity. 
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ORGANIZATION NAME GEOGRAPHY 
Institute of Tropical Medicine Antwerp Belgium: Flanders and Brussels Capital Region 


KEY PARTNERS 


KU Leuven, Institut Pasteur, London School of Hygiene and Tropical 
Medicine (Vaccine Confidence Project) 


FOCUS POPULATION 


Healthcare workers (including students in healthcare) 


METHODOLOGY TYPE 


Mixed method design: in-depth interviews, focus group discussions, SOCIAL MEDIA PLATFORM/S USED 


social media analysis and online surveys | Yy @ (Discord) 


Healthcare workers (HCW) are a priority group for COVID-19 In the context of 

vaccination and important sources for lay vaccine decisions. Yet, polarization and vaccine 
considerable vaccine hesitancy exists among HCW whilst experienc- mandates, discussing 

ing pressure to vaccinate, leading to unspoken hesitancy. We strive COVID-vaccination has 

to restore Belgian HCW vaccine confidence by listening to concerns become problematic 

using hybrid social listening and by building an online safe space to among healthcare workers 
enable an engaging, empathic dialogue. in Belgium. Creating safe 


spaces for dialogue among 
healthcare workers is a 


Our study confirmed healthcare workers (HCW) often experience multi-faceted effort that 
difficulties talking about COVID-vaccination with peers and patients. requires adapting to the 
Vaccine confident participants felt unvaccinated colleagues were changing pandemic context, 
quite defensive or that it was a lost cause trying to convince them. offering multiple modes 

On the contrary, vaccine skeptical participants felt they were often of communication, having 


considered egocentric, and their concerns not taken seriously. 
Hesitant healthcare workers also noticed that vaccine confident 
colleagues would sometimes speak harshly about unvaccinated 
patients, which made them reluctant and fearful to disclose their own 
vaccination concerns. 


active dialogue moderation, 
preventing an echo chamber 
of misinformation, and 
putting aside the aim of 
increasing vaccination 
uptake to promote trust. 
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exposure to potential misinformation. Based on that 
experience, the research team decided to bring ina 

professional facilitator to moderate the live dialogue 
sessions, which turned out to be an overwhelmingly 


Most participants regretted the strong polarization 

in society (especially among HCW) regarding COVID- 
vaccination and appreciated our project's idea to 
develop a safe space for dialogue. Despite many 


participants claiming it was “too late” for dialogue, most 
participating HCW preferred to speak to peers face-to- 
face rather than on a text-based social media platform. 
While anonymity was confirmed to be important, the 
ability to see each other offers more authentic dialogue. 


Unexpectedly, several participants perceived the 
research project itself to be an effective venue for 
dialogue. The opportunity to express vaccine-related 
concerns to peers or to researchers during any of the 
research activities helped them overcome frustrations. 
Vaccine confident participants mentioned they enjoyed 
participating but felt less need to be in dialogue with 
other HCW to alleviate frustrations. 


KEY PROCESS INSIGHTS 


The project was impacted by two important factors: (1) 
the Belgian COVID-epidemic and (2) the positionality of 
the researchers. 


After almost two pandemic years and considerable 
pandemic and vaccination fatigue, it was difficult to 
motivate HCWs to participate in our project. In addition, 
the project took place during the fourth (Delta) and fifth 
(Omicron) wave in Belgium and HCWs were burdened 
with yet another challenging and busy time at work. The 
team had to act fast and be innovative in finding new 
ways to recruit participants, such as through Facebook 
ads, labor organizations and universities. 


Our position as researchers in public health institutions 
also impacted our ability to co-create an online safe 
space with hesitant HCW. It was a challenge to find the 
right balance between openly sharing our personal and 
professional vaccine-related thoughts with participants 
while maintaining neutrality, both of which were 
important to establish trust between researcher and 
participant. We also struggled finding the right balance 
between letting people openly voice their concerns 

and avoiding creating anti-vaccination echo chambers, 
which had the risk of increasing vaccine hesitancy 

in HCW. As some participants hijacked focus group 
discussions to voice their opinions (e.g. monopolizing 
speech, consistently interrupting others, being rude or 
threatening towards those with different opinions), the 
research team was confronted with the difficult trade- 
off between encouraging open dialogue and minimizing 


positive experience for participants. 


RECOMMENDATIONS 


Leverage social media to foster vaccine dialogue, trust in 
democratic science-making and vaccine confidence by: 


Listening to vaccine narratives and addressing 
vaccine concerns & information deficits 


Passive social listening: use qualitative and data 
science methods to continuously characterize 
online vaccine sentiments, identify vaccine and 
vaccination concerns & information deficits 


Conduct / support active research (surveys, 
interviews, FGDs): 


Share listening results with public health stakehold- 
ers for adapted communications and interventions 


Creating venues for dialogue and peer support 


Scaling up knowledge of the impact of dialogue on 
vaccination confidence building and wellbeing 


Creating a dialogue facilitation workforce (dialogue 
is a craft that cannot be fully automated): 


Creating vaccine dialogue venues that would cater 
to different needs: 


Organize live vaccine-related interaction sessions 


Create a text-based venue to interact on vaccina- 
tion, with different channels: 


Creating peer experience support groups 


As soon as the hesitant participant 
expressed her doubt, the flow of the 
dialogue changed drastically. Other 
participants congratulated her for being 
courageous enough to express her doubt 
here. They then began sharing about how 
they had doubts themselves and how they 
too did not always agree with all the policy 
decisions surrounding the vaccination 
campaign. At the end of the focus group 
discussion, the vaccine hesitant partici- 
pant expressed that she had “regained her 
faith in humanity” thanks to this dialogue. 
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ORGANIZATION NAME GEOGRAPHY 
International Longevity Centre UK 
Lead Researcher: Sophia Dimitriadis, Senior Economist, ILC 


KEY PARTNERS 
Digital Willow 

Vision One 

Forster Communications 
Open Age 

Panelbase 


YuGo METHODOLOGY TYPE 
The Coalition for Life-Course Immunisation 


Desk research, focus group interviews, online survey and test 
campaigns. 


FOCUS POPULATION 


SOCIAL MEDIA PLATFORM/S USED 
Our target groups included those less likely to take up routine 
vaccination, older people living in deprived areas of the UK, and f 
older people from certain ethnic minorities (particularly people with 
Black African and Black Caribbean backgrounds). We also explored 


whether younger people could influence older people's vaccine 
uptake through social media. 


This project explored whether younger generations can encourage older New research from 
people to get vaccinated. Specifically, ILC created and tested social media the International 
content to explore whether engaging younger people on social media Longevity Centre-UK 
increases uptake of the flu and pneumococcal vaccines among older people (ILC) finds that social 
living in socioeconomically deprived areas. ILC also explored whether results media advertising can 


differ among people from Black-African and Black-Caribbean communities 
within these areas. 


be a cost-effective 
way of engaging older 
people in deprived 

Our social media advertising campaigns may have cost effectively communities with 
improved flu vaccination rates among deprived communities — and the vaccination. 
pneumococcal adverts generated vaccine booking link clicks at a cheaper 

rate than flu adverts. 
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Our campaigns received significant engagement. 
Engagement metrics were far higher than average for 
healthcare adverts: over 75% of users who saw the 
advert engaged, and the reactions and survey responses 
indicate that most users perceived them positively. 


Targeting older users directly on social media gave 
better trackable results than targeting younger audienc- 
es — but we can't rule out the effectiveness of engaging 
younger audiences for offline conversations. 


Younger people were more likely to click on the adverts 
to learn more than older users, but were then less 

likely to act — although we couldn't track all potential 
follow-up actions. 


Although most advert reactions were positive, the 
majority of comments on the adverts were negative, 
indicating that our campaigns resonated more with 
the undecided than antivaxxers—but these negative 
comments may have increased engagement. 


Our target communities claim to struggle to see their 
General Practitioner (GP) or book a vaccination appoint- 
ment — health practices can make this easier 


KEY PROCESS INSIGHTS 


As more older people, including those from marginalised 
groups, take to social media, this channel offers a real 
opportunity to address these inequalities. 


But younger people are still more likely to use social 
media and to engage with its content. As evidence 
suggests that they can influence the health behaviour of 
older relatives — and as we know that older people tend 
to use social media to keep up with friends and family — 
these younger users could be part of the solution. 


But this hasn't been tested before on social media, our 
study sought to try this out. 


We found that targeting older users directly on social 
media gave better trackable results than targeting 
younger audiences. Despite stereotypes to the contrary, 
social media campaigns appear to be able to effectively 
engage marginalised older adults with vaccination and 
increase uptake. 


Our findings also suggest that younger generations 
were engaged with our campaign, and by some mea- 
sures, more engaged than older adults. Our findings 
suggest they may have generated impact offline- we 
just could not completely track this. As younger 
generations are less expensive to reach on social media 
compared to older generations, there is clearly scope 
for future research to dig into this question further. 


Have You Discussed The Jab Yet? 


INCREASING JABS 


Screen capture from a social media campaign from 
the International Longevity Center exploring intergener- 
ational relations in increasing routine vaccinations. 


A 
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PHOTO: INTERNATIONAL LONGEVITY CENTRE 


RECOMMENDATIONS 
Build the evidence base and scale up findings 


The Department for Health and Social Care and the 
NHS should: 


e Increase investment in social media campaigns 
to increase the uptake of routine vaccination 
(especially less well-known vaccines, such as the 
pneumococcal and shingles vaccines) among 
older people living in deprived areas in the UK. 


Measure the impact and cost-effectiveness of 
future social health campaigns 


The Department for Health and Social Care and the 
NHS should: 


e Analyse and publish the results of future social 
campaigns on vaccination and collaborate where 
relevant, to maximise learnings/impact and 
prevent under-investment. 


Vv DO YOUNGER RELATIVES PASS ON HEALTH 
INFORMATION? IF SO, HOW? 
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E Most deprived quintile 


Address knowledge and accessibility barriers to 
vaccination 


The NHS and the Department for Health and Social 
Care should: 


e Offer the NHS pneumococcal vaccine in 
community pharmacies. 


e Ensure that all GP practices send reminders and 
consistently discuss pneumococcal vaccination 
with eligible patients. 


e Create a single online hub where people can book 
all routine vaccination appointments and display 
these options prominently on the NHS website. 


Explore further ways to use data gathered by social 
media for public good 


Policy makers should: 


e Explore ways to encourage social media owners 
to share data with government health systems or 
researchers while mitigating the risk of negative 
consequences — including the data being exploited. 


Test unanswered questions from our study 
Health policy makers should: 


e Explore whether using social media to engage 
younger family members is a cost-effective way 
to increase vaccination uptake among older 
family members. 


65% 
55% 
44% 44% 
35% 


No - they do not NET Yes - influence NET Yes - pass on 
pass on health 
messages to me 


behaviour health messages 


BAME communities 


ORGANIZATION NAME 
IRD Global 


GEOGRAPHY 


KEY PARTNERS 


Government of Sindh’s Expanded Program on Immunization 


FOCUS POPULATION 


Low-income communities in three high-risk districts in Karachi, 
Pakistan 


METHODOLOGY TYPE 


Districts East, South, and Korangi - Karachi, Pakistan 


SOCIAL MEDIA PLATFORM/S USED 


O f Yy TikTok Snack Video 


Participatory Mixed Methods Research 


Pakistan has faced multifaceted challenges with respect to the COVID-19 pan- 
demic and its vaccines. Misinformation, conspiracy theories, lack of awareness, 
disbelief in possible health ramifications, and limited access to equitable services 
have been identified as impediments in vaccine confidence. 


Our program is a community-led behaviour change campaign that utilizes partic- 
ipatory action research for user-informed social media engagement to improve 
COVID-19 vaccine confidence and promote vaccine uptake in three high-risk 
districts of Karachi, Pakistan. We developed six online groups across two social 
media platforms, supplemented through interactive community engagement 
enabling communities to co-create content for digital campaigns addressing 
social barriers around COVID-19 and its vaccine. Vaccine camps were also held in 
these districts in collaboration with the Government of Sindh to provide equitable 
COVID-19 vaccine services coupled with routine immunization. 


In low resource 
settings, bridging 
online engagement 
with participatory 
on-ground 
activities amplifies 
community voices 
and experiences 
resulting in 
increased public 
interest in health 
topics and an 
uptake in vaccines. 
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KEY FINDINGS 


IRD initiated project Boost employed participatory 
engagement in three high-risk districts in Karachi, 
Pakistan, to build user-informed social media 
campaigns around COVID-19 vaccine confidence. From 
October 2021 to June 2022, the project leveraged six 
online platforms (WhatsApp, Facebook, Instagram, 
Twitter, TikTok, and Snack Video) to capture diverse 
audiences from the identified districts and ran multiple 
online campaigns, utilizing user-informed content, 
based on social barriers faced by communities during 
the pandemic. The project supplemented these online 
efforts via immersive on-ground activities, such as 
through Theatre of the Oppressed performances, health 
education, and various below-the-line activations, 

via trained Field Officers and a cohort of volunteer 
grassroots leaders. 


Six online groups were developed (mixed gender on 
Facebook and women-only on WhatsApp) with 1,425 
members, stimulating dialogue and dissemination. 
Moreover, the on-ground team engaged 5,756 people 
across the catchment areas. Through these efforts, 
1,039 people availed COVID-19 vaccines through mobile 
camps held in the selected districts. It was noted that a 
strategy which bridged online engagement with on- 
ground activities highlighting the efficacy of vaccines 
to restore normalcy in a post-pandemic world garnered 
immense public interest and uptake of vaccines. Thus, 
participatory approaches embedded in outreach efforts 
ensure inclusion and reach of otherwise neglected 
voices and narratives in-person and through various 
social media. 


KEY PROCESS INSIGHTS 


COVID-19 fatigue, parallel vaccine roll-outs, and 
relaxations in restrictions dominated the contextual 
landscape for the duration of our project. In light of 
changing public mood, we shifted our primary strategy 
from vaccine science communication to lifestyle-based 
narratives focused on social barriers caused by the 
pandemic. 


The rhythm of urban life also pushed us to pivot our 
on-ground strategy from repeated engagement with 
grassroots leaders to increasing awareness via health 
education sessions. Our identified grassroots leaders 
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became our focal point and our bridge to communicate 
effectively with the target communities. These grass- 
roots leaders proved extremely helpful in the execution 
of on-ground activities, including mobilization efforts for 
the vaccination camps. They also emerged as opinion 
leaders of the projects’ digital community groups. 


The use of macro-influencers in the initial phase of 

the campaign proved less fruitful in comparison to 

local micro-level content creators in the latter phases. 
This shift was also made possible by timely feedback 
gauged through participatory community engagement. 
An inclusive collaborative approach made it possible to 
iterate the project implementation strategies in a timely 
fashion. Continuous community feedback aided in 
creating a momentum in engagement levels both online 
and on-ground. 


Provision of COVID-19 vaccines along with routine 
childhood immunization via mobile camps was an 
effective outreach and access strategy, as it enabled 
reaching difficult-to-engage populations. By integrating 
services, we offered a one-stop-shop for people's 
vaccination needs. In addition, collaborating with the 
government in camp execution also added legitimacy 
to the online and on-ground campaigning. 


RECOMMENDATIONS 


We recommend acknowledging local communities’ 
experiences around COVID-19 and developing com- 
munication strategies that a) address gaps made by 
public health specialists during the pandemic, b) take 
into consideration the role of on-ground community 
networks in supporting digital dissemination, and c) 
utilise participatory approaches in online and on-ground 
engagement. 


Women are key agents for influencing behavior change 
at homes, but have limited access to social media and 
public spaces. The utilization of on-ground engagement 
and development of women-only local WhatsApp 
groups signify how strategies can easily be made 
inclusive and equitable. 


Traditional data collection methods, such as surveys, 
and in-depth interviews may not yield honest results 
in politically-charged and controversial areas such as 
COVID-19 where government mandates were used 
as means to ensure vaccine uptake. This may have 


in fact done more harm towards vaccine confidence; while communities may take up the vaccine, they will not 
trust public health experts if information is not shared transparently. It was noted in this project that while being 
recorded, participants often answered questions in a positive light, but revealed hesitations when the recorder was 
off. Therefore, it is important to incorporate alternative sources of data collection methods such as narrative field 
reports, observation rubrics, and other qualitative metrics. 


Moreover, it is important to note that developing online communities organically in low-resourced environments 
takes time. To expedite this, having offline activities that are tangible, such as theatre performances, art installa- 
tions, and on-ground mobilization, linked to online platforms is an effective strategy. 
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ORGANIZATION NAME 


Johns Hopkins University 


GEOGRAPHY 
Odisha, India 


Rajiv N. Rimal, Professor, Johns Hopkins Bloomberg 
School of Public Health 


KEY PARTNERS 
Johns Hopkins University 
Loyola University 

D-COR Consulting 


Swasthya Plus 


ai a ealtadl ae SOCIAL MEDIA PLATFORM/S USED 


ie ® 


FOCUS POPULATION 
Young Adults (18-35) in Odisha, India 


METHODOLOGY TYPE 
Mixed Methods: 


Online experiments 
Qualitative interviews 
Social network analysis 
A/B testing on Facebook 


This project had five components. First, with the help of Indian actors, eight short videos 
(differing on collectivistic or individualistic appeal, humor, and protagonist gender) were 
created and their effectiveness tested online. While vaccine skepticism increased, use of 
humor appeals stemmed this tide most successfully. Second, using social network analysis 
methods, comments on the vaccine videos on YouTube were analyzed based on the polarity 
of sentiment. Visualizations indicated that YouTube videos do not exist in a vacuum; rather 
they are part of a subscriber and recommender network, which is important in understanding 
how videos are disseminated to audiences. Third, qualitative interviews revealed that the 
quality of characters’ acting abilities can also spill over to the message they advocate; thus, 
actors who provoke audiences with anti-vaccine messages should be chosen with care. A 
fourth component laid out the issues media production interventions should consider when 
projects require a balance between the rigors of scientific control, on the one hand, and 
artistic creativity, on the other. Finally, media made for television were adapted for social 
media platforms in various ways (truncating the beginning, adding subtitles, converting the 
rectangular aspect ratios to square dimensions, etc.) and then were subject to A/B testing 
on Facebook. Adapted media garnered more views and greater engagement overall, thus 
highlighting their value. 


Vaccine messages garner skepticism while vaccination messages are found to be more 
acceptable. Focusing on the benefits of actually getting the vaccines appears to be more 
effective than focusing on the vaccines themselves. When done correctly, humor-based 
appeals appear to confer two tangible benefits: they garner attention by raising the message 
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Vaccine 
messages 

can increase 
skepticism 
among 
viewers, but 
well executed 
storylines, with 
the use of humor, 
can reduce 
skepticism. 
While the use 
of humor ina 
pandemic may 
be tricky, it can 
also increase 
attention and 
diffuse tension 
to boost 
engagement. 


above the clutter of other health messages in the 
information ecosystem, and they reduce defensiveness by 
diffusing the tension. 


For vaccine messaging, it is important to consider the 
creative approach and the dissemination platform, as the 
platform's algorithm determines who is likely to receive the 
message. Effective search engine optimization can mitigate 
the limitations of recommended systems, ensuring that 
content does not get caught in exclusionary algorithms. 


In this research, we have developed an approach that 
makes algorithmic outputs visible and offers a means 

to describe them through visual analysis and content 
analysis of the comments posted by viewers. Through this 
approach, we conclude that effective message design can 
facilitate communication of health-oriented information 
beyond the “home” channel to be “seeded” into other 
channels extending the reach of the messaging. 


Understanding how platform algorithms work is an 
important takeaway for health communicators designing 
social media campaigns for vaccine acceptance. 


KEY PROCESS INSIGHTS 


This research created eight videos similar in all ways 
except one, with videos either having a male or a female 
lead, being serious or humorous, and with a collective 
safety appeal or an individual safety appeal. Achieving 
similarity across videos while also ensuring stark 
contrast in manipulated attributes required collaborative 
engagement across languages and contexts, with the 
video language being Odia, and targeting rural audiences 
in Odisha, India. Scientific and artistic partners had to 
rethink their processes and orient the cast, crew, and 
other stakeholders with the distinct research needs. 


This process included the involvement of members of a 
Community Advisory Board that provided feedback on 
the script development process. To do this, professional 
audio versions of the scripts were developed to test their 
effectiveness and appropriateness, and feedback was 
incorporated before developing the final videos. Involving 
the community — in a process that selected people from 
diverse walks of life — provided constant and meaningful 
feedback for the creative and research team. 


Much of public health videos produced by Indian health 
agencies are created for traditional media. To make them 
more appealing on social media, a number of measures 


need to be taken, as revealed through A/B testing 
conducted on Facebook for this project. These include 
shortening the length, including subtitles throughout, 
and changing the rectangular aspect ratio to a square (to 
accommodate viewing on cellphones). Doing so may be 
tedious, but they garner greater engagement. 


RECOMMENDATIONS 


Effective health communication balances the old and the 
new: sound conceptual ideas that have guided research 
and practice for many years with rapidly-evolving and 
algorithm-driven information landscape that can greatly 
enhance reach and outreach. Health communicators 
have known for a long time that effective communication 
requires understanding the audience; tailoring 

messages according to audience members’ needs and 
expectations; using credible and trustworthy sources to 
disseminate messages; and being sensitive to cultural 
nuances that affect message interpretation. These 
considerations remain as important as ever. 


In recent times, thrown into this mix have been 
technology-driven algorithms that take into account 
audience members’ prior behaviors, affiliations, and 
viewing habits to shape who is served which information. 
Although much of this is shaped by audience members’ 
trace on the Internet, a good portion of it is also shaped 
by how messages are titled, which keywords are invoked, 
and how channel characteristics are optimized for 
maximum engagement. Not paying adequate attention to 
these factors will almost certainly constrain the reach and 
impact of health communication efforts. 


Our research shows that it is possible to deliver high- 
caliber videos created according to the cultural and 
cognitive background of the message recipients. In 
addition, the study sheds light on when and how to 

use humor in vaccine messaging, and it illustrates that 
audiences make clear distinctions between how they 
view vaccines, on the one hand (with suspicion and 
skepticism), and how they view vaccination, on the other 
(with much positivity). Careful use of humor can greatly 
improve attention and message receptivity. 
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ORGANIZATION NAME GEOGRAPHY 
Jonathan Yong Lee (Individual) - Stanford University USA 


FOCUS POPULATION 


Twitter users 


METHODOLOGY TYPE 


Machine learning-based topic modeling 


SOCIAL MEDIA PLATFORM/S USED 


y 


The CDC faces a growing challenge of maintaining public trust in the face of constant Rhetoric 
attacks on their trustworthiness from a variety of sources across all relevant media related to the 
channels. Yet, little is known about the specific types of rhetorical strategies aimed at politicization 


supporting or undermining trust in the CDC. This project takes a step towards characteriz- of the CDC 

ing such rhetorical strategies by applying an unsupervised, machine learning-based topic may have 
modeling algorithm on a corpus of Twitter posts (i.e., “tweets”) related to trust in the CDC. played the 

The corpus of tweets was filtered based on keywords that include variants of the word biggest role in 
“CDC” (e.g., CDC, CDCgov, @CDCgoy, etc.), “trust” (e.g., trust, trustworthiness, trusted, etc.), a ‘ 

and “credibility” (e.g., credibility, credible). Using the topic modeling algorithm, the corpus driving distrust 
of approximately 230,000 tweets were categorized into “subtopics,” each referring to a ml the CDC 
particular content matter (e.g., tweets about the COVID vaccine, tweets about Biden, etc.). early In the 
Drawing on interdisciplinary theoretical frameworks which broadly define perceptions pandemic. 
of trustworthiness and credibility as being composed of perceptions of (1) expertise/ 

competence and (2) integrity/benevolence, subtopics of tweets specifically relevant to 

these components of trustworthiness were further categorized into broader topics. The 

relative changes in prevalence of these broader topics were evaluated over a period of 

time spanning the advent of Twitter to the date of the collection of the corpus of tweets 

(March 21, 2022). 
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KEY FINDINGS 


“Subtopics” of tweets related to the two theoretical 
components of trustworthiness and credibility 
(expertise/competence & benevolence/integrity) were 
grouped into broader “topics.” As seen in Table 1, 
“Ambiguous” are tweets that provide a reason to trust or 
not trust the CDC; however, the reasons given generally 
fail to specify whether it relates to the CDC's expertise/ 
competence or benevolence/integrity. For instance, 
flip-flopping is one of its contained subtopics; however, 
accusations of flip-flopping do not clearly relate to 
expertise/competence or benevolence/integrity. The 
topic of “Expertise/competence’ refers, as expected, 

to tweets that specifically argue the CDC is or is not 
trustworthy on the basis of expertise/competence. Of 
note, the vast majority of these tweets are in Support 

of the CDC — suggestive of evidence that the CDC’s 
trustworthiness is rarely attacked for lacking expertise/ 
competence. The “Benevolence/integrity” topic refers to 
arguments for/against CDC benevolence/integrity. This 
topic is further divided into ambiguous, nonpolitical, and 
political — respectively referring to whether the tweets 
suggest the CDC has nonpolitical ulterior motives (e.g., 
ties to pharma, profit, patent exploitation), political 
ulterior motives, or unspecified, ambiguous motives. 


Evaluating the relative prevalence of these broader 
topics before and during the COVID-19 pandemic, 
Figure 1 shows evidence that the relative prevalence of 
“Benevolence/integrity (political)” tweets dramatically 
increased at the beginning of the pandemic compared 
to the year prior to the pandemic. Indeed, nonpolitical 
concerns (e.g., pharma ties) significantly decreased 
relative to political concerns. This suggests that the 
significant drop in CDC trust early in the pandemic may 
have been especially driven by 

concerns about the CDC having 


v 


KEY PROCESS INSIGHTS 


Machine learning-based topic modeling technology 

has made incredible progress in recent years, and over 
the course of this project, various older and newer 
algorithms were trialed, including latent dirichelt analysis 
(LDA) and the use of jointly embedded topic, document 
and word vectors. In the end, the use of jointly embedded 
vectors appeared to strike a better balance between 
having too many topics with sufficient specificity vs. too 
few topics with diluted/insufficient specificity. 


RECOMMENDATIONS 


The preliminary results of this project highlight the 
potentially outsized role that rhetoric related to the 
politicization of the CDC may have played in driving 
CDC trustworthiness and credibility concerns, even 
relative to other concerns such as CDC ties to phar- 
maceutical companies, which are commonly cited as 
drivers of CDC distrust. Further research is required to 
elucidate causal pathways linking politicizing rhetoric 
to public distrust of the CDC; however, the large relative 
prevalence of this type of rhetoric on Twitter seen in 
this analysis suggests that public health institutions 
would be wise to think carefully about how it can 
communicate with the public in a way that affirms that 
its public health recommendations are based purely on 
scientific evidence as opposed to political concerns. 
More broadly, these findings add to increasing concerns 
that worsening political polarization is threatening the 
ability of public institutions to effectively guide and 
govern the public through crises. 


RELATIVE PREVALENCE OF TOPIC'S TWEETS OVER TIME 
Timeline of levels of trust in the CDC by sentiment. 


been politicized and other 
political partisan concerns. 
Meanwhile, concerns about 
CDC expertise/competence 
remained relatively small in 
comparison to benevolence/ 
integrity concerns before and 
after the start of the pandemic. 
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ORGANIZATION NAME 
Laboratory of Data Discovery for Health (D24H) 


GEOGRAPHY 


Singapore 
KEY PARTNERS Thailand 
The University of Hong Kong 


Saw Swee Hock School of Public Health, National University of 
Singapore and National University Health System 


Health Intervention and Technology Assessment Program, Ministry 
of Public Health, Thailand 


Department of Infectious Disease Epidemiology, London School of 
Hygiene and Tropical Medicine 


Department of Digital Health and Innovation, World Health 
Organization, Switzerland 


FOCUS POPULATION 


Vaccine hesitant parents of children younger than 12 years old in 
Hong Kong, Singapore, and Thailand. 


Children of vaccine hesitant senior citizens over 60 years old in 
Hong Kong and Thailand. 


METHODOLOGY TYPE 
Randomized Controlled Trial 


The primary objective of this project is to develop and evaluate a conversa- 
tional Al service on communication platforms for users to obtain accurate 
vaccine information and to assess the veracity of vaccine information in 
Hong Kong (China), Singapore, and Bangkok (Thailand). 


Randomized controlled trials (RCTs), social listening analytics, and the 
RE-AIM framework will be employed for formative, impact, and process 
evaluation of the intervention. 


This project will be conducted by a consortium of partners including Hong 
Kong University, Vaccine Confidence ProjectTM (VCP), National University 
of Singapore, and the Thai Ministry of Public Health, Health Intervention 
and Technology Assessment Program (HITAP). 
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Hong Kong Special Administrative Region, China 


SOCIAL MEDIA PLATFORM/S USED 
Facebook Messenger 


This first multi-site RCT on 
the impact of conversational 
Al services (chatbots) on 
attitudes towards vaccination 
suggests that deploying 
vaccine chatbot might offer 

a promising intervention 
option that is feasible and 
effective in improving vaccine 
confidence and acceptance, 
while also improving 

access to vaccine-related 
information. 


KEY FINDINGS 


From Hong Kong, Singapore, and Thailand, a total 
of 616 persons completed the pre- and post- 
intervention evaluations, of whom 255 persons 
(41.4%) were assigned to the intervention group 
and used the chatbot. In terms of pre- and post- 
intervention responses, compared with the control 
groups, Hong Kong chatbot-users reported a greater 
improvement in their access to “information about 
COVID-19 vaccine on children” (control group 

vs. chatbot-users: 14.9% vs 27.6%). However, 

in Singapore, chatbot-using parents reported a 
higher increase in the perceived susceptibility of 
“COVID-19 infection for their children in the next 6 
months” (20.8% vs. 48.3%) and a greater decrease 
in the perceived vaccine safety: “Overall, | think 
COVID-19 vaccines are safe for children” (6.3% vs. 
31.0%). In Thailand, a higher proportion of chatbot- 
users reported a significant improvement in their 
perception on “COVID-19 vaccine effectiveness 

for children against all variants” (21.7% vs. 30.4%) 
and “COVID-19 vaccine effectiveness for children 
regardless of the manufacturer of the vaccine” 
(21.7% vs. 28.6%). We did not detect association 
between chatbot use and the likelihood of being 
influenced by misinformation about COVID-19 
vaccines. However, in view of meaningful findings 
such as statistically significant improvements in 
chatbot users’ perceptions of vaccine effectiveness 
and vaccine information accessibility across study 
sites, we saw the positive potential of vaccine 
chatbots as public health interventions in the 
vaccine-hesitant populations. 


KEY PROCESS INSIGHTS 


Key issues revolving around vaccine hesitancy can 
emerge, re-emerge, and spread in any population for 
a myriad of complex reasons including complacency 
against the threat of vaccine preventable diseases 
(VPDs), lack of confidence in vaccine safety and 
effectiveness, distrust in health authorities or political 


leadership, inconvenient access to vaccine information 
and vaccines, and exposure to vaccine misinformation. 


Hello, is there anything | can help you 
with? 22 4:10 


If I experience side effects after the 
COVID-19 vaccination, will there be 
any compensation for my loss? 

SF ANNY 


The government has set up an 
Indemnity Fund for Adverse Events 
Following Immunization with 
COVID-19 Vaccines to provide 
financial assistance to eligible 
persons who experience serious 
adverse events after the COVID-19 
vaccination. Detailed information on 
AEFI Fund can be found on this 
website: https:// 
www.covidvaccine.gov.hk/en/ 
AEFI_Fund. Learn more about 
similarities and differences between 
side effects and adverse events 
following immunization (AEFI): 
https://www.covidvaccine.gov.hk/en/ 
faq. De 4:11 


Á AUTOMATED WHATSAPP RESPONSE 


Conversational chatbot on WhatsApp for users to 
obtain accurate vaccine information and to assess 
the veracity of vaccine information in Hong Kong 
(China), Singapore, and Bangkok (Thailand). 


To mitigate COVID-19 vaccine hesitancy, the novel 
chatbot solution was developed through social 
listening findings and formative consultations with 
existing COVID-19 chatbot developers, and was aimed 
to enhance infodemic management and vaccine 
confidence and uptake. 


In the process of conducting this research, the chatbot 
was continuously adapted based on participant 
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feedback and real-time monitoring of COVID-19 
vaccine-related news, mandates, misinformation, 
rumors, and concerns. For example, vaccination of 
children was on the horizon during the study period, and 
this policy meant changing or adding another group 

to the target population. This continuous monitoring 
and evolution allowed the chatbot to remain timely and 
relevant. The chatbot was also able to disseminate 
accurate COVID-19 vaccine-related information to the 
public in an easily accessible format from a credible 
source. Chatbots allowed instantaneous responses 

to pressing questions, which might not be feasible 
with other forms of health communication (e.g., 
conversations with a healthcare worker). 


RECOMMENDATIONS 


This is the first multi-site RCT on the impact of 

vaccine chatbots on attitudes towards vaccination. 

It was conducted during Omicron outbreaks, when 
governments were ramping up local vaccine campaigns 
and targeted the most hesitant people, whose 
dependent family members had rejected or delayed 
COVID-19 vaccination until after January 2022. 


Given the prevalent use of social media in Asia and 
across the world, and the rising issue of infodemics, 
this study suggests that deploying conversational 
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Al services (chatbots) as an intervention might offer 
a promising option that is feasible and effective in 
improving vaccine confidence and acceptance while 
improving access to vaccine-related information. 


Vaccine chatbots could complement existing tools 
and offer an invaluable information source that can 
supplement other forms of health communication and 
engage users who might otherwise feel judged when 
asking vaccine-related questions. Additionally, social 
listening is critical for keeping a finger on the pulse of 
real-time changes in sentiment, concerns, rumors, and 
misinformation, as this monitoring allows chatbots to 
evolve and remain relevant. 


This study targeted the most hesitant population, 
who had delayed or refused COVID-19 vaccines 

for months. Chatbots might help identify the main 
reasons for postponing or rejecting the vaccines, and 
a targeted communication and engagement strategy 
will be needed to address the issues. More evidence 
is encouraged to assess if chatbots can effectively 
respond to concerns, rumors, and misinformation 
before they gain traction and can be utilized as a 
communication channel complementary to current 
public health strategies. 


GEOGRAPHY 
Western Cape, South Africa 


ORGANIZATION NAME 
Macro-Eyes, Inc. 

Monika Obrocka 

Megan Rogers 

Jean Githae 

Mesay Bejiga 

Jeni Stockman 


FOCUS POPULATION 


All (population of Western Cape Province) 


METHODOLOGY TYPE 
Machine Learning model 


SOCIAL MEDIA PLATFORM/S USED 


With the knowledge, tools, and technology we have developed working on predicting 
demand for health services such as immunisation, we have learned that health seeking 
behaviors change depending on what is required to access services, and available supply. 
Drawing data from social media, we will build and test a machine learning system to 
predict local vaccine hesitancy. These predictions will equip the health system with 
actionable insights on where to prioritize allocation of vaccines and target resources to 
communities where behavioral interventions are needed before doses are sent. 


Through this research, we have found that the use of social media to monitor and respond 
to public sentiment around vaccines is a valuable tool for health officials and allows 

health officials to get an accurate picture of public sentiment surrounding vaccines. 

This information can help precise allocation of resources during vaccination campaigns, 
including how and when to roll out vaccinations to specific populations. In addition, we 
have discovered that using specific platforms (such as Facebook) for this purpose is more 
effective than others (such as Twitter). This information could be used by public health 
officials to develop targeted outreach campaigns in order to improve vaccination rates 
among at-risk populations. 


Learning sentiment through leveraging of public social media data can also help identify 
any areas where there may be opposition or hesitancy towards getting vaccinated, so 
that these issues can be addressed directly by health officials. Additionally, tracking 


Investigating 

the connection 
between social 
media and 
vaccine uptake is 
important to help 
organizations 
and leaders 
understand how 
social media 
platforms can 

be leveraged 

to increase 

the health and 
resiliency of 
communities 
around the world. 
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social media chatter provides a way for health officials 
to quickly address any concerns or misinformation 
that may arise online about vaccines. Doing so helps 
ensure that people are accurately informed about the 
safety and benefits of vaccinating themselves and their 
families. 


Our analysis found that both sentiment analysis 
(measuring public confidence) and spatial accessibility 
(measuring ease of access) are important indicators for 
understanding why some communities may be hesitant 
about getting vaccinated. 


KEY PROCESS INSIGHTS 


Over 6 months (January — June 2022) we developed 
a model using sentiment analysis and incorporating 
derived data on access to infrastructure to assign 
numerical hesitancy scores to geographical areas. 


To better understand individual intent to get vaccinated, 
we considered practical and logistical barriers to vac- 
cination. More specifically, ease of access to vaccines 
and vaccination sites which play a role in vaccine hes- 
itancy. For example, if people do not have easy access 
to a vaccination site, they may be more likely to hesitate 
or refuse vaccinations. This approach considers factors 
such as capacity of facilities; the cost of the clinical visit 
and vaccine; and physical access including geographic 
and functional proximity to vaccines. The approach 
also has the added benefit of identifying areas with low 
accessibility and potential intervention points. 


We have developed a model framework that combines 
two approaches, namely sentiment analysis and 
access to infrastructure, to assign numerical scores 
representing vaccine hesitancy to geographical areas. 


Number of posts 


0 


neutral negative 
Sentiment 


Source : Sentiment analysts of vaccine relevant Facebook posts 


positive 
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We combined sentiment analysis and data on access 
to infrastructure to ensure that the outputs distinguish 
between low vaccination rates due to hesitancy and 
due to limited access to vaccination sites. 


RECOMMENDATIONS 


The use of social media to monitor and respond to 
public sentiment around vaccines is a valuable tool for 
health officials. This methodology needs to be applied 
and run-in real time during the roll out of vaccines, or 
during specific vaccine campaigns, specifically when 
there is more likely to be social media engagement. The 
main COVID-19 vaccination programs in South Africa 
have passed their peak in terms of rollout and imple- 
mentation. However, this methodology has potential for 
future, routine vaccination programs, whether COVID- 
related or other, as well as being implemented from the 
start during future pandemics. 


Presented methodology allows health officials to get an 
accurate picture of public sentiment surrounding vac- 
cines. This information can help them make decisions 
about how and when to roll out vaccination campaigns. 
It can also help identify any areas where there may be 
opposition or hesitancy towards getting vaccinated, so 
that these issues can be addressed directly by health 
officials. Additionally, tracking social media chatter 
provides a way for health officials to quickly address 
any concerns or misinformation that may arise online 
about vaccines. Doing so helps ensure that people are 
accurately informed about the safety and benefits of 
vaccinating themselves and their families. 


BREAKDOWN OF SENTIMENT IN 
FACEBOOK POSTS 


Among the posts identified as being 
relevant, the majority were of positive 
sentiment, as shown in the figure below. We 
hypothesized that as this is a retrospective 
scrape, looking back several months to 
when the vaccine roll out was actively 
taking place, the negative or very polarizing 
posts may have been removed by group 
moderators or taken down by the poster 
themselves. Additionally, we only scraped 
public pages and groups. We recognize that 
more emotive and polarizing posts could 
be taking place in private groups which we 
do not have access to. This will remain a 
constraint. 


ORGANIZATION NAME GEOGRAPHY 
Mali Health Bamako, Mali 


KEY PARTNERS 
OSI Tech 


FOCUS POPULATION 


Women living in peri-urban communities with limited reading and 
writing skills 


METHODOLOGY TYPE 
SOCIAL MEDIA PLATFORM/S USED 


This project used a mix of qualitative and quantitative evaluation es 
methods, including individual interviews and focus group discus- Keneya Blon 
sions. The project relied and built on a participatory evaluation 

of social and gender norms conducted in the target community 

conducted before the project. 


This project 


Kènèya Blon is a voice-based platform developed with women living in 
peri-urban communities in Bamako, Mali, for delivering health information in demonstrated that the 
low-connectivity and limited-resource settings. This project aims to test the use of participatory 

ability of Kenéya Blon to improve vaccination knowledge and confidence in methods and tools to 
marginalized communities. develop and disseminate 


voice-based social media 
messages improved 
The key results of this project include: both knowledge of, 
and confidence in, 
vaccination against 
COVID-19 in underserved 
peri-urban communities 
in Bamako, Mali. 


e 100% of women who used the application demonstrated improved 
knowledge about the benefits of vaccination against COVID-19 


e 75% of women who used the application expressed confidence in 
vaccination against COVID-19 


e 73% of women who used the application shared the information they 
learned with others 


e 60% of women who used the application felt better equipped to con- 
vince others to get vaccinated against COVID-19 
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KEY PROCESS INSIGHTS 


Using participatory methods to understand the needs 
and behaviors of the target population, and the norms 
which govern them, was essential to the ability of the 
project to have an impact on vaccine confidence. The 
end users were involved in every step of developing 
the original platform; when assessing how to use it to 
address vaccine confidence as a part of this project, 
their participation was necessary. 


At the beginning of the project, 95% of those surveyed 
in peri-urban communities in Bamako did not trust 
COVID-19 vaccines and had no intention of getting 
vaccinated. According to the same survey, the primary 
reason given was a lack of access to trusted health 
sources that could provide accurate information, or 
correct misinformation. A lack of access to health 
professionals and reliable health information was a key 
factor in acting on misinformation and the adoption of 
positive behavior changes. 


The Kénéya Blon application was able to remove this 
primary barrier. Users were able to access health 
personnel for the management of their concerns 
related to COVID-19 vaccination, which is why despite 
the short duration of the intervention of this project it 
obtained significant results. 


RECOMMENDATIONS 


e Technology and digital interventions must be 
adapted to the realities of each community or 
population it is trying to serve; this adaptation can 
be based on the very design of the application, 
the language used, the images used, but also the 
social/gender norms, the kinds of misinformation 
circulating, etc. 


e When coordinating with offline health services, 
ensure the quality and availability of a respondent 
for interactions with users, as well as the quality 
and availability of the vaccination service at the 
health center level; when possible, train these 
providers in the use of digital tools/messages being 
used in their communities 


e The dissemination of messages relating to 
COVID-19 must be ongoing 


e Integrate the use of relevant, local digital tools into 
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mobilization strategies around vaccination against 
COVID-19 


Promote the use of digital tools within target 
communities with on-the-ground and face-to-face 
strategies to build trust 


Though access to technology is increasing, it will 
continue to be a limiting resource for millions, 
especially for women with limited literacy and 
numeracy skills: continue the search for strategies 
that can share voice-based messages on the types 
of phone and technologies that are available 


“I am not against vaccination in general, but 
in relation to vaccination against COVID-19, 
| received a lot of contradictory information 
from my friends or through social networks; 
which created doubts and uncertainties 
that led to hesitation and even fear of the 
COVID-19 vaccine on my part. 


My fear started to dissipate when | installed 
Kénéya Blon in my phone and received 

the correct information. | then made the 
decision to get vaccinated after discussions 
that | had the opportunity to have with the 
health staff through the application. | have 
already gone to the health center to register 
for the vaccination when it is available. | am 
just waiting for the health center to call me 
for an appointment to get vaccinated.” 


- Mme. Bathily, Sabalibougou, Mali 


Á BUILDING TRUST 


Local participants learn about the Sabalibougou app at 
a promotional event held by Mali Health 22/2/2022 in 


the community of Sabalibougou in Bamako, Mali. 


ORGANIZATION NAME 
Minority Rights Group 


KEY PARTNERS 

Bytes for All 

Verité Research Pvt Limited 

Grand Synergy Development Initiative (GSDI) 


FOCUS POPULATION 


Minority and marginalized communities in the target countries. 


Algeria—focus on Amazigh/Tamazight speakers. 


Kenya-focus on Somali population. 


Pakistan-focus on religious minorities. 
Sri Lanka—focus on Tamils and Muslims. 


GEOGRAPHY 
Algeria, Kenya, Pakistan, Sri Lanka 


SOCIAL MEDIA PLATFORM/S USED 
ZIS 
aunt 
=x nyO 
BFA wane f v 
k 
Se nyA 
esearci ka 
IN 
GSDI aunt Y 
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METHODOLOGY TYPE 


Partnership: MRG signed an MOU with three partners with existing 
strong relationships of trust and acceptance in target communities. 
Each partner led on the work in their respective country with 

MRG leading in Algeria. Furthermore, B4A led on the social media 
monitoring technical aspects and MRG coordinated the overall and 
extracted lessons from each bulletin to share with international 
policy makers, media and influencer audiences. 


Theory of Change: Considering social media content analysis 
can surface variations in vaccine confidence and access between 
minority and majority communities. Social media content based 
on these findings and created with participation can address 
confidence sappers and differential allocation of resources for 
vaccination provision. 


Research: The project evaluates information gathered in each 
context with two methods. The first method involved the use of 
social media listening and monitoring tools, CrowdTangle and 
Brandwatch. The second method served to triangulate the findings 
from monitoring social media and was different for each setting: a 
largescale face-to-face survey in Sri Lanka involving 2479 partic- 
ipants, 30 radio talk shows in various locations in Kenya, thirteen 
citizen journalism pieces from different parts of Pakistan, and four 
focus group discussions (FGDs) with participants inside Algeria or 
involving diaspora representatives. 


The findings of this study are the result of social media monitoring 
(Facebook & Twitter) and analysis, covering a period of more than 
12 months starting from each countries’ vaccination rollout or 
kick-off vaccination campaigns and to date (around January 2021— 
March 2022). Each country had its own timeframe of analysis: 

Sri Lanka January: January — October 2021, Algeria, and Kenya: 
January 2021 - March 2022, Pakistan: February — March 2022. 


Overall, a sample of 2,727 Facebook comments (and tweets only in 
Sri Lanka), were analyzed with respect to four sentiments: COVID-19 
vaccine confidence, reservations towards the vaccine, access to 
vaccines; and distrust in health and other government authorities 
regarding the administration and dissemination of information 
related to the vaccine. 


Data was compiled and analyzed to create two bulletins x target 
country. All bulletins were produced in multiple language versions 
(Tamazight + French/Arabic, English + Somali, English + Sinhala + 
Tamil, English and Urdu) to ensure that all the relevant communities 
could have access to it. 


To address lower confidence in minority communities the team 
established participatory processes to investigate media messag- 
ing to address confidence sapping rumors or beliefs (e.g., in Sri 
Lanka and Kenya). In both cases, messages were disseminated and 
promoted to reach target audiences, in Pakistan, a meeting with 
government representatives was planned, and in Algeria, a webinar 
to address the findings. 


Equity: This project actively targeted marginalised communities 
known to be at risk of lower rates of vaccination to address inequity. 
Marginalised communities are not homogenous and discrimination 
affects women, people living with a disability, the elderly as well 

as other vulnerable sub-groups of the target communities were 
considered wherever possible. 


VCF IMPACT REPORT | 


| 85 


PROJECT SUMMARY 


This project will use Brandwatch and CrowdTangle social 
media monitoring to track and understand online sharing 
across diverse ethnic, religious and linguistic communities 
about COVID-19 vaccine confidence, uptake and access 

in Algeria, Kenya, Pakistan and Sri Lanka. Due to partial 
penetration of social media use among some poor/remote 
communities, the social media data will be triangulated 
using one other method in Algeria, Sri Lanka and Pakistan. 
In each context, uptake is affected by supply, access and 
vaccine confidence. The social media monitoring will 
search for sentiments relating to frustration concerning 
lack of supply as well as expressed grievances concerning 
disparities in supply or selection of vaccines. Based on the 
findings, the research team will publish quarterly briefings 
and will generate locally relevant social media materials 

in minority languages to address rumors, disinformation, 
mistrust and/or problems in access or supply of vaccina- 
tions. The ongoing social media monitoring will continue 
to ascertain the impact of those materials and their dis- 
semination and sharing across the community in question. 
The project will focus on indigenous Tamazigh speakers 
in Algeria, Muslim Somalis in Kenya, religious minorities 

in Pakistan and Tamil, Muslim, and Sinhala Buddhist 
ethno-religious groups in Sri Lanka. Each of these contexts 
involves communities whose members are politically, 
socially, and economically marginalised, and other than 
Pakistan, living in remote and rural locations. The research 
will use MRG's and partners’ unrivalled knowledge of, 
access to and trust of these excluded communities to 
understand and report on their vaccination confidence, 
uptake, and access. 


KEY FINDINGS 


e When data is disaggregated by ethnicity, language 
and religion, findings help explain issues such as 
residual pockets of non-vaccination in Sri Lanka, or 
the fivefold difference in vaccination rates between 
different areas of Kenya. 


e Tamil and Muslim social media users were more 
likely to display low to no confidence, and based on 
the survey, three times more likely to remain unvac- 
cinated in comparison to members of the majority 
Sinhala community. 


e Amazigh social media users in Algeria were more 
likely to express no confidence in COVID-19 vaccines 
than the general population (28 per cent compared to 
18 per cent). 
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When data is disaggregated potentially 
significant differences emerge which 
may help to explain residual pockets 
of non-vaccination in Sri Lanka, or 

the fivefold difference in vaccination 
rates in different areas of Kenya, 

thus treating diverse populations as 
homogenous in a misguided attempt 
to save time or money or to promote 
efficiency which can create a high risk 
of back-firing and can mean efforts do 
not reach the intended targets. 


e Muslims in Kenya were significantly more likely to 
express that the vaccine was not a real vaccine 
(17 per cent versus 4 per cent among Christians 
and Somali), a reservation unique to Kenya and not 
mentioned in the other three countries. 


e Among the Amazigh community in Algeria, conspir- 
acy theories were the most frequently mentioned 
reservation: 25.3 per cent versus only 13 per cent of 
social media users from the general population. 


e Access to information in all languages in Algeria 
demonstrated some limitations, as official online 
sources did not provide information on COVID-19 
vaccines in Tamazight, an official language in Algeria, 
but made it equally available in Arabic and French, 
the latter not being an official language. 


e Data captured via social media did overall display 
differences compared to triangulation methods, 
which we consider are likely due to the bias in the 
sample of the population (smart phone owning 
and active on social media) as well as differences 
in frankness and openness between online and 
real-world expression settings. 


KEY PROCESS INSIGHTS 


Having minority representatives working on the research 
was crucial for the success of our project. As a way of ex- 
ample, the recruitment of Amazigh researchers provided 
a strong knowledge of the context, necessary language 
skills and access to the community, which is significant 

in a context where security concerns are high. 


Our methodology can be adapted to various contexts, 
yet social media research can be challenging due to local 
factors. This was the case in Algeria, where the use of 
Tamazight in social media is not widespread, especially 


in the context of health crisis and this limits chances 
of collecting comments from the Amazigh community 
based on language. Besides, some Amazigh people 
have Arabic names, and this means some members of 
the Amazigh community were not identified in the data 
by name. 


In Pakistan, the unavailability of sufficient data on our 
topic may be due to the low proportion of religious 
minorities within the overall population and the fact 
that they may be less likely to own and regularly use a 
mobile phone. Therefore, having it complemented with 
on ground data from the survey and citizen journalism 
pieces was a valuable part of the research. 


It was essential to shift the research from posts on 
social media to comments. Most posts were from 
official sources and did not indicate any sentiments 
towards COVID-19 vaccines, whereas comments to 
posts revealed a wide array of sentiments. 


Focusing solely on social media findings is not 
representative in settings that have partial internet 
penetration, and thus such data should always be 
supported with additional sources of information from 
non-digital sources as our project tested and achieved. 


RECOMMENDATIONS 


e Everyone involved in rolling out COVID-19 vacci- 
nations should be aware of ethnic, religious, and 
linguistic barriers to vaccine access and take up. 


BELIEFS AND 
MISCONCEP- 
TIONS ABOUT 
REPRODUCTIVE 
AND SEXUAL 
SIDE EFFECTS 
CAUSED BY 
COVID-19 
VACCINES 


Breakdown by 
ethnicity/religion in 
Sri Lanka based on 

survey data 


> I Strongly disagree | Somewhat disagree 


Hill Country Tamil PZE 


Sinhala MELA 


vaccine leads to sterility 


Hill Country Tamil eye 
Sinhala KEKS 


Sri Lankan Tamil BELA 


Sri Lankan Tamil 52% 22% 


“It is very discouraging waking up in 
the morning with an intention of saving 
the population from the pandemic 
through administering vaccines but 
returning with almost 60% of our daily 
target completed and unused doses.” 


-Nurse at a government health facility on Marsabit 
County, northeast Kenya 


e To ensure reach to all groups, the benefits, risks, 
and modalities of vaccinations should be commu- 
nicated in all languages spoken in a territory, via 
multiple media outlets, via multiple formats (online, 
offline, written, audio), and through means that can 
reach areas not served by phone signals or internet 
access, and for those who are not literate. 


e Where distrust exists between local populations 
and health authorities, extra efforts should be made 
to involve trusted figures to convey key information 
and help rebuild trust. 


e Availability and take up of vaccinations should be 
monitored and tracked by a wide range of char- 
acteristics including geography, ethnicity, gender, 
age, and disability. 


e Provision of community sympathetic online report- 
ing systems concerning clinics not functioning, 
stock outs or other health access or health dignity 
problems that don't depend on individuals owning a 
mobile phone. 


Not sure IB] Somewhat agree [fj Strongly agree 


vaccine leads to sexual dysfunction 


32% 12% 


48% 18% 


42% 16% 


17% 


23% 


13% 
9% 
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ORGANIZATION NAME 
Morning Star Lodge 


GEOGRAPHY 


Implemented virtually within Star Blanket Cree Nation, 18 kilome- 


ters Northeast of Balcarres, Saskatchewan. 


KEY PARTNERS 
Star Blanket Cree Nation 


Kihew Kawaskasit Health Services 


FOCUS POPULATION 


Reserve-residing and urban members of Star Blanket Cree Nation. 


METHODOLOGY TYPE 


Indigenous community-based participatory research. 


SOCIAL MEDIA PLATFORM/S USED 


Community-run information page on Facebook. 


This project will pilot two social media interventions that are informed by historical and 
contemporary sources of hesitancy. One intervention will address the narrative tactics 
used by conspiracy theories while the other employs behavioural insights. Social media 
analytics will provide intermediate measures of effectiveness. Established Indigenous 
Community Research Advisory Committees will guide all aspects of research and provide 
assessments and recommendations. 


Indigenous vaccine hesitancy does not necessarily mean low vaccination rates. Assuming 
that Indigenous vaccine hesitancy will result in low vaccination rates is deficit-based and 
undermines the strengths, abilities, and expertise within a community. 


In the complex space of Indigenous vaccine confidence, many factors, such as culture, 
can be used to overcome vaccine hesitancy or justify refusal. Furthermore, the community 
members engaged in this research do not perceive empathy to be a consistent 
component of vaccine confidence promotion. 


Vaccination-promoting social media posts that are entirely developed by community 
appear to perform better than those that are, partially or entirely, developed externally. 
While the sample size was small, the six posts developed, and piloted, during this research 
had a mean of 0.66 likes and 46.166 views. The six community-developed posts that most 
closely preceded this research had a mean of 2.33 likes and 64.66 views. 
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In Canada, 
Indigenous 
vaccine 
hesitancy 

is complex; 
however, as is 
often the case, 
solutions may 
be more likely 
found within 
the community 
rather than 
within the 
Academy. 


Finally, history does contribute to Indigenous vaccine 
hesitancy; however, Indigenous Peoples exist within 
modern Canada and are exposed to, and influenced 
by, many of the same factors leading non-Indigenous 
Peoples to be COVID-19 vaccine hesitant. A failure 

to account for these modern factors leaves our 
understanding of Indigenous vaccine hesitancy 
incomplete. 


KEY PROCESS INSIGHTS 


Communities are dynamic, and this includes their 
priorities. Something identified as high priority when 
funding is applied for may not remain so when funding 
is approved and research begins. Furthermore, the 
piloting of any solutions must include what the commu- 
nity intends to do. Many have argued that the solutions 
to Indigenous health inequities lie within Indigenous 
communities, knowledges, and cultures. Therefore, 
funding proposals for health research with Indigenous 
communities should be written such that research can 
pivot from seeking for solutions, to supporting and 
seeking to understand Indigenous-led self-determined 
solutions. 


Finally, it is well Known that research with Indigenous 
Peoples needs to be self-determining. That being said, 
self-determination and research feasibility exist on a 
spectrum with research becoming less feasible with 


increased self-determination. Researchers need to be 
aware of this and select, and accurately report, exactly 
where their research sits on this spectrum. 


RECOMMENDATIONS 


Anyone looking to promote vaccine confidence within 
Indigenous communities in Canada should first look 
to support self-determination and community-driven 
solutions. Communities have expertise and local 
cultural and social knowledge that are crucial for 
effective vaccination promotion. 


Supporting community, however, is not the first step in 
promoting Indigenous vaccine confidence. Before being 
able to support community, mutually beneficial and 
authentic relationships must exist between community 
members and healthcare workers or health researchers. 
Forming and strengthening these relationships should 
be a priority so that they are in place before the next 
major public health crisis emerges. 


Finally, an academic understanding of vaccine confi- 
dence and Indigenous vaccine hesitancy can be useful; 
however, what is actually happening in community 

can differ greatly from academic reporting. Ultimately, 
observations and comments emerging from communi- 
ty appear more useful and accurate than those that are 
found in academic literature. 


‘Ww COMMUNITY DEVELOPED POSTS OUT PERFORMED PILOTED POSTS 
A chart depicting the performance (views) of community developed posts in comparison to piloted posts. 


@ Community Developed @ Piloted Posts 


80 


60 


40 


Views 


20 


VCF PRELIMINARY IMPACT REPORT | | 89 


ORGANIZATION NAME GEOGRAPHY 
Oklahoma State University USA 

Dr. Skye Cooley, Oklahoma State University 

Dr. Juwon Hwang, Oklahoma State University 

Dr. Asya Cooley, Oklahoma State University 

Dr. Robert Hinck, Air University 

Dr. Jared Johnson, Oklahoma State University 

Pablo Aguirre, AV8 Analytics 


FOCUS POPULATION 


Hispanics in the United States 


SOCIAL MEDIA PLATFORM/S USED 


METHODOLOGY TYPE B 


Mixed-method (depth interview, thematic and computational social 
media analyses, experimental survey) 


The goal of this research project was to address the disproportionate 
impact of COVID-19 among marginalized migrant populations, as well 
as the lack of research on vaccine confidence among these popula- 
tions. This novel interdisciplinary mixed-methods study design sought 
to identify, model, analyze, design, and validate narratives of vaccine 
confidence among marginalized migrants in the US. 


The initial phase of the project featured in-depth interviews with 
marginalized migrants in the Stillwater, Oklahoma area, and surround- 
ing region, to qualitatively explore the themes of vaccine confidence. 


The second phase included qualitative data analysis of social media 
comments expressive of vaccine confidence as well as a quantitative 
narrative analysis, identifying the specific features and story structures 
migrants use when expressing vaccine confidence. 


Finally, the third phase used the previous analyses to design narratives 
expressing vaccine confidence. These narratives were post-tested 
through a nation-wide survey. 
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This study demonstrates 
how sourcing culturally 
specific narratives 
enhances message 
persuasiveness; here, 
confidence in the 
COVID-19 vaccine 
among Hispanic survey 
participants is increased 
when participants are 
exposed to culturally 
specific expressions 

of vaccine confidence 
derived from Hispanic 
communities. 


KEY FINDINGS 


The findings show the importance of narrative theory 
when crafting persuasive messages and validate that 
culturally specific narratives and imagery are more 


persuasive than generic messaging to target audiences. 


An important consideration in communicating to 
vulnerable Hispanic migrant populations is allowing 
them to see and hear representations of themselves in 
the messaging. Particularly in messaging concerning 
public health. Here cultural specificity is not merely 
adopting the symbolic aspects of culture and layering 
them into messaging, rather it is an amplification 

of organic community narratives with key mutually 
understood cultural nuances. While preference is also 
given for messages with images that look like self, 
that more persuasive narratives accentuated Hispanic 
cultural themes of collectivism and community 
concern is the study’s key finding. These themes were 
consistent drivers in expressions of vaccine confidence 
among Hispanics in data collected across the project. 
The study also shows those at high risk, and thus 
perhaps more likely to be seeking out information 
concerning the vaccine, responded most favorably to 
culturally specific narratives. Overall the study shows 
the significant impact culturally tailored messaging can 
have in increasing confidence in the vaccine to those 
seeking information in Hispanic communities. 


KEY PROCESS INSIGHTS 


The study highlights the value of mixed methodology 
approaches to crafting persuasive narratives, as well as 
further validates the value of narrative theory. Our novel 
methodology and empirical findings demonstrate the 
effectiveness of identifying culturally situated discours- 
es aS ameans to amplify organic community narratives 
to boost vaccine confidence. 


Given the barriers to access the 
Hispanic migrant community in the US 
face concerning COVID-19 vaccination, 
this study shows that amplifying 
organic cultural narratives and pro- 
viding representation of Hispanics 

in messaging can enhance vaccine 
confidence; particularly among those 
who see themselves at higher risk to 
COVID-19. 


RECOMMENDATIONS 


Our study shows that messaging toward Hispanics 
concerning vaccination should consider amplifying 
organic cultural expressions of vaccine confidence 
from within the community. Having access to medical 
information outlining the benefits and risks of vac- 
cination and hearing from individuals trusted in the 
community such as nurses is important to Hispanics. 
The responsibility to family and community members 
resonates. We would recommend COVID-19 messaging 
directed toward Hispanics take into account existing 
community expressions of vaccine confidence and 
amplify those as a priority, as well as to meet Hispanic 
community members where they are with easily 
accessible information. 
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EXPLORING DRIVERS OF HPV VACCINE CONFIDENCE AND COMMUNITY- 
LED SOLUTIONS LEVERAGING THE EXPERIENCES OF MOTHERS, 


DAUGHTERS, AND HEALTH-CARE PROVIDERS IN BRAZIL 
(a 


ORB International: Quantitative survey of 1,000 interviews with mothers through focus 
groups, discussions with mothers, in-depth interviews with HCPs, 


Zonnny Heald and co-recreation of workshops with mothers and their daughters. 


Richard Hooper 
Michael Gibson 


Rohan Puri 
Brazil 
Vaccine Confidence Project at the 
London School of Hygiene and Tropical Medicine: 


Sarah Malycha 
Clarissa Simas 
Elliot Grainger 
Ana Bolio 

Felipe Pires Dias 


Real Chemistry: 


Eleanor Dickinson 
Henri Ghosn 


Mothers of daughters who are eligible for a HPV vaccine 


PROJECT SUMMARY 


Quantitative research project seeking to develop recommendations and 
insights on how to improve HPV vaccine uptake. Focused on Brazil for its large 
population, high rates of HPV among sexually active, unvaccinated women 
(Wendland et al., 2020), and its struggle to contain COVID-19 online misinfor- 
mation; though the approach and findings will be designed to be replicable for 
other markets. Our project utilized methodology was designed to provide rich 
actionable insight through four research phases. Each research phase has 


provided a new perspective on the challenges to HPV vaccination uptake and 
provides insight into the development of solutions to overcome them. 
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KEY FINDINGS 


The key barrier identified to HPV take-up is a lack of 
clear information not hesitancy. Brazil has a strong 
vaccination tradition that has somewhat insulated the 
population from any spikes in anti-vaccine sentiment or 
vaccine hesitancy. Post COVID-19 there is also a greater 
interest in vaccines and demand for more information. 
Trust in the health system and health care providers 
(HCPs) is high, HCPs are the most trusted source of 
information on vaccination. There is a lack of clarity 
around HPV vaccination, our research participants 

were unclear on availability and benefits of vaccination. 
Information on HPV vaccination, particularly concerning 
the immunization window and vaccination timings, is 
patchy and inconsistent. The journey to jab is unclear 
and inconsistent, depending on your local health provider, 
access to vaccination and information can vary wildly. 
Though social networks are a vital source of information 
for our audience there is suspicion over information 
received through these sources. To counter this mistrust 
the right people should be used to deliver the messages. 
HCPs were seen as ideal for this task. Cancer prevention 
is the most motivating messaging for mothers. Mothers 
and their daughters want to see themselves reflected in 
communications. 


KEY PROCESS INSIGHTS 


e Collaborating with our cross-disciplinary team 
of partners has worked well in the design and 


Persona’s attitudes 
to HPV vaccines 


te 


Hate "hE ene imde nme 
ee eee liol 


implementation of our initial phases of research, 
learning from each other's fields of expertise. 


e Data protection law in Brazil (LGPD) meant we 
were unable to collect social media handles of 
research participants and for them to share screen 
shots prior to the focus groups as planned. We 
addressed this situation by asking participants to 
send screenshots of online material they saw in the 
run up to the focus groups to our local fieldwork 
partner. The screenshots were then collated and 
sent to us, without any identifying information, to 
inform our research and analysis. 


RECOMMENDATIONS 


e Increasing HPV vaccination rates in Brazil will not 
be easy but the communications challenges we 
face are relatively simple to overcome. 


To win we need to get the fundamentals of com- 
munication right, before we consider innovation, we 
need to create a framework for communications. 


e We need to grow awareness around the HPV 
vaccination to a wide audience. 


e We need to provide clear, concise and credible 
information to help our audience understand why 
vaccination is vital. 


e We need to reinforce messaging with timely and 
relevant reminders of when, where, and how to 
access vaccination. 


WILLINGNESS TO 
SEEK HELP ON 
SOCIAL MEDIA 


A comparison of 
willingness to get 
help from social 
media pages 
featuring Hispanic 
image and Spanish, 
Hispanic image 
and English, 
wires non-Hispanic 
— image and Spanish, 
acre and non-Hispanic 
image and English 
by participants’ 
ethnicity. 


o eg ee mee a et ee ee 
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ORGANIZATION NAME GEOGRAPHY 
Purpose Campaigns LLC Nigeria, South Africa, India 


KEY PARTNERS 
Mythos 
SBM 


FOCUS POPULATION 


Members of Public WhatsApp Groups in Nigeria, South Africa and 
India 


Members of private civil society WhatsApp Groups in Nigeria 


SOCIAL MEDIA PLATFORM/S USED 


METHODOLOGY TYPE ® B y 


The study was conducted between the 17th of February and the 

3rd of March 2022 by posting specifically designed pro-vaccine We also used email. 
content into open WhatsApp group channels in India, Nigeria 

and South Africa, and evaluating the impact of that content on 

both WhatsApp group behavior, cross-channel behavior within 

WhatsApp, and cross-platform sharing behaviors. This approach 

was compared to other communication strategies including 

posting in closed WhatsApp group channels as well as email and 

paid media dissemination. 


This work was conducted in India, Nigeria, and South Africa in February and March 2022, 
and used an open WhatsApp group channel approach with adapted methods. In addition 
to open WhatsApp groups, this study used closed WhatsApp groups, paid media, and 
owned email lists. The purpose of this experiment was to test approaches to encourage 
sharing of pro-vaccine content pieces - both sharing cross-channel (between open and 
closed channels within WhatsApp), and sharing cross-platform (between email, paid 
media, and WhatsApp). 


Public WhatsApp groups provide rich territories for seeding pro-vaccine content, however 
the platform is used in distinct ways in different contexts and content must be adapted for 
the platform. 


e Open WhatsApp group channels provide rich territories for seeding pro-vaccine 
content. 28% of groups engaged with a pro-vaccine content link. Over 5% of the 
content links initiated in public group channels were re-shared within WhatsApp to 
other WhatsApp channels one or more times. Seeding content into WhatsApp can 
generate clicks and shares to public social media platforms such as Facebook. 
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Public 
WhatsApp 
groups provide 
rich territories 
for seeding 
pro-vaccine 
content, 
however the 
platform 

is used in 
distinct ways 
in different 
contexts and 
content must 
be adapted for 
the platform. 


e Links that were initially shared on WhatsApp were 
also more likely to be reshared on WhatsApp than 
other origin platforms. 


e Paid media and email lists can also generate 
content sharing into WhatsApp. However, the rate 
of cross-platform sharing was lower than sharing 
within WhatsApp. 


e Funny pro-vaccine content was more likely to be 
clicked-through, engaged with, and shared than 
more ‘official’ pro-vaccine content via paid media and 
email platforms. This was particularly true in South 
Africa and Nigeria. 


e Indicative results suggest that audiences were more 
likely to share the meme style pro-vaccine content 
on the paid media platform, however, once users 
reached the more formal Sharer Tool page from the 
Meta platform. Nigerian and South African users 
were more likely to share the official content. 


e Content in email or WhatsApp performs better with 
aCTA. Indicative findings suggested that on these 
platforms, content with a CTA is more likely to be 
shared from these channels, particularly in India. 
This was also true in Nigeria, but not in South Africa. 


e On paid media, share rates were typically higher on 
platform channels for content without a CTA. 


SHARES 
Re-share rate FROM 
WHATSAPP 
WHATSAPP 
9.80% 4.76% 0.65% CHANNEL 
Re-shares on 
call to action 
and official 
posts. 


Funny CTA 


Sa 


KEY PROCESS INSIGHTS 


A high volume of WhatsApp users can be reached at 
scale. Our research teams were able to identify over 

fifty thousand open WhatsApp group channels in four 
specific countries, targeting groups associated with a 
range of specific interests from politics to football and 
alumni groups that all provide opportunities to both 

learn about and develop content for audiences with clear 
associative ties. 


Nevertheless, the process of joining these groups is time 
consuming and challenging. Yield rates of viable groups 
are relatively low (roughly 1%) and manual group joining 
can be cumbersome. However, automation tools can 
provide capacity to reach large numbers of WhatsApp 
users at scale by automating the group joining and 
evaluation process. 


Another challenge to highlight is the attrition of 
WhatsApp groups over time, as they may change names, 
become administrator only, or remove the researcher 
account. Any programme relying on interventions in 
WhatsApp groups must continuously identify and join 
new groups to mitigate against this challenge. 


Because of challenges related to group joining, more 
long-term research is needed to confirm our findings as 
only a small number of groups were included. 


A high proportion of Nigerian WhatsApp groups 
were excluded from our sample because of our strict 
research protocols that removed any groups where 
explicit material was exchanged. Campaigners could 
utilize less restrictive screening protocols to enable 
access to a higher volume of users. 
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RECOMMENDATIONS 


Utilizing open WhatsApp group channels is a 

key modality for interventions targeting closed 
WhatsApp group channels. If a campaign aims to 
tackle misinformation or engage audiences within 
WhatsApp, it is beneficial to seed content within the 
platform itself. 


Campaigners targeting Indian audiences should 
prioritize WhatsApp as a key platform. On 
WhatsApp, Indian users were the most frequent 
sharers of pro-vaccine content. On other platforms, 
Indian users were the least frequent sharers 
compared to Nigerian and South African users. This 
is particularly pertinent given the finding that mis- 
information levels were the highest in open Indian 
WhatsApp groups when compared to Nigerian and 
South African groups. 
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Campaigners targeting Nigerian audiences should 
also prioritize WhatsApp and funny content. 
Whilst click-through, engagement, and share rates 
were lower among Nigerian audiences compared 
to those in India, they did demonstrate significant 
engagement on WhatsApp and when funny, meme 
style content was used. 


Campaigners targeting South African audiences 
can expect significant click-through and en- 
gagement on paid media platforms as well as via 
email. Our study found that South African users 
were the most likely to click-through both from the 
Meta platform to our Sharer Tool landing page and 
from the email campaign to a share post on their 
app. However, South African users who clicked 
through from the Meta platform became less likely 
to share than Nigerian and Indian users. This result 
could indicate a preference for less official/more 
informal spaces among South African Meta users, 
though further investigation is required. 


ORGANIZATION NAME GEOGRAPHY 
RMIT University Primary: Vietnam 
Abdul Rohman, PhD. RMIT U Vietnam Secondary: Indonesia 
Dyah Pitaloka, PhD. Monash U Malaysia 
Erlina Erlina, PhD. Gajah Mada University Indonesia 
Duy Dang, PhD. RMIT U Vietnam 
Ade Prastyani, MD, MCHM. Independent researcher 


FOCUS POPULATION 


People with Disabilities 


METHODOLOGY TYPE 


Participatory, action research. 


SOCIAL MEDIA PLATFORM/S USED 


The research was conducted remotely using Facebook and Zoom 


Our project is a direct response to the limited availability of vaccine-re- 
lated information for people with disabilities (PwD) during the COVID-19 
pandemic in Vietnam and Indonesia. Through action, participatory 
research, the project in particular advocates for more equitable vac- 
cination programs. Through a series of online and offline information 
sessions and interviews, the expected research findings have the 
potential for creating equitable hybrid information hubs for groups who 
have been historically marginalized and to mainstream the need for a 
more inclusive pandemic preparedness, response, and recovery plans 
in the Global South. 


Although confident about the 
vaccine, many people with 
disabilities still face systemic 
barriers to vaccinations; 
more equitable information 
materials and dissemination 
is needed to promote 
vaccinations for people with 
disabilities within pandemic 
responses. 
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KEY FINDINGS 


The following findings are based a preliminary under- 
standing on 150 interview data collected through a 
participatory action research from September 2021 to 
February 2022 in Vietnam and Indonesia: 


e The limited availability of specific vaccine-related 
information deprives PwD of lacking information 
pertaining to side effects and the safety of the 
vaccines. If the information is available, its presenta- 
tion was inequitable, primarily to the Deaf and Blind 
communities. The vaccination registration form was 
not designed for diverse users. No columns provided 
for PwD to state their disability type. 


¢ Social media platforms in general were not the main 
information source about vaccines. But, in Vietnam 
Facebook and Zalo were among the platforms 
that the Deaf community perceived to be useful 
for seeking vaccine-related information from other 
PwDs. Those who live in rural areas or locations with 
an unstable Internet connection relied on opinion 
leaders and traditional communication means. 


e — |nequitable vaccination sites became a barrier to 
vaccinations. 


e The wheelchair users found the vaccination sites 
in their neighborhood did not have ramps or were 
in places with multiple stairsteps. The Blind found 
it hard to navigate the vaccination sites as there 
was no accessible signage. The Deaf often faced 
difficulties in communicating their pre-existing 
health conditions due to the absence of sign 
language interpreters at the vaccination site. 


e Inconsistent support at the local level was another 
challenge PwD faced. Due to their health condi- 
tions, doctors were reluctant to vaccinate them at 
the neighborhood vaccination site. PwD were often 
to get vaccinated at a referred hospital, therefore. 
During the strict lockdown where many public 
transportation services were halted, going to the 
hospital was arduous. Despite that, Zero to limited 
support was given by the local authorities although 
PwD was considered as a priority list in the national 
vaccination program. 


GOOD TO GO 
A person with Disabilities receiving a vaccination. 
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KEY PROCESS INSIGHTS 


Working with members of communities with disabilities 
have created opportunities to better engage the groups 
who have been historically marginalized and gather 
insiders’ perspectives. We engaged the communities 
from the proposal development, implementation, to 
evaluation stage of the project. This practice mitigates 
the extractive tendency of researchers, preference 

to top-down interventions, and ableism in research 
intersecting technologies and disabilities. In diverse 
but digitally divided environments such as Vietnam and 
Indonesia, this practice gives a more nuanced view as 
to what digital and social media platforms are useful 
for communicating the intervention, what information 
sources fit the worldviews of the communities we are 
focusing on, and how to create equitable information. 


In a context where the intervention is given to com- 
munities live in a highly centralized government role, 
working with stakeholders affiliated with government 
bodies is crucial. In Vietnam, we worked with two med- 
ical experts affiliated with a national vaccine institute 
and the ministry of health for direct information sharing 
about the country’s vaccination programs. 


In places where the implementation of disability rights 
is inconsistent, working with organizations of PwD 
allows for embedding the intervention into the existing 
infrastructures. Rather than creating a brand-new 
infrastructure, our project embedded the intervention to 
six partnered community organizations with adequate 
abilities to reach PwD living in both urban and rural 
areas with different Internet access and affordability. In 
small towns and rural areas, the information sessions, 
as a part of the action research, we had organized were 
the only intervention dedicated to providing vaccine-re- 
lated information for people with disabilities (PwD). 


PS 


RECOMMENDATIONS 


Vaccinations, from vaccine information to vaccination 
registration, should be equitable to people from all 
walks of life, including people with disabilities (PwDs), 
whom the WHO classifies as being at a higher risk of 
severe disease and death if contracted COVID-19. 


Ideally, the government should collaborate with local 
organizations such as organizations of people with 
disabilities and services involved in the vaccination 
rollout to identify hurdles inhibiting PwDs from getting 
a vaccine, as well as devising effective approaches to 
addressing hindrances. Messages about vaccinations 
should entail vaccine disability-and-morbidity-specific 
safety and efficacy details, instructions for COVID-19 
self-treatments; in addition to the importance of 
disability rights during the pandemic. The information 
must be presented in a range of accessible formats, 
e.g., sign language, braille, pictorial illustrations, and 
through diverse channels, e.g., social media, traditional 
means of communication, such that to accommodate 
all types of disabilities or those residing in rural areas. 


Physical accessibility is a cardinal principle in choosing 
vaccination centers, apart from the informational role. 
There should be numerous vaccination locations readily 
available. Sites should be furnished with accessible 
facilities, e.g., ramps, handrails, and ensure that health 
and vaccination staff are armed and trained to assist 
PwDs walking through the immunization procedure. 
With the aforementioned solutions, vaccine confidence 
and uptake within the disabled communities are likely 
to increase substantially. 


DISABILITY-INCLUSIVE COVID-19 
VACCINE INFORMATION 


Vaccine-related information must be 
accessible and communicated through 
equitable communication approaches 
to reach individuals with a range of 
disabilities. 


THE INFORMATION 


Risks facing people with 


disabilities (PwDs) 


Differences between 


and 0 4 additional and booster 


importance of vaccination doses 


How the vaccine works 0 5 How to get vaccine-related 


Vaccine's effectiveness 
and safety for PwDs 


Deafness/hearing 
impairment 


Videos with caption, 
sign language 
interpretation, 

telephone lines include 
text-messaging option 


VACCINE REGISTRATION 


Provide digital and non-digital 
formats: mobile apps, webs, 
live-support phones or paper- 
based forms 


Include a section specifying 
the type of disability and 
accommodation required by 
PwDs 


information 


0 6 Available accessibility 


features at the vaccination 
sites 


Visual Cognitive 
impairment disability 
Audios, braille, Easy-to-read 

screen-reader text, plain 
accessible, image alt language, 
text, large and high- pictorial formats 

contrast fonts 


VACCINATION CENTER 


Present vaccination process 
through: walk-throughs, maps, 
signages, audio or video 
descriptions 


Provide on-site information 
support: sign language, 
communication notes, markers 
or pens and papers 


COMMUNICATION CHANNELS 


Television 


Urban areas Areas with 
with good low internet 
internet connection 
connection 


Loudspeaker 


Written communication 
and direct outreach 


Information sources: authoritative health experts, 
organizations of people with disabilities (OPDs), 
local community leaders 
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TUNAJENGANA: USING SOCIAL MEDIA TO MOBILIZE YOUNG EAST 
AFRICANS TO CHAMPION COMMUNITY UPTAKE OF COVID-19 VACCINES 


vaccinated and are now advocating for COVID-19 vaccination 


: in their families and/or communities. 
Shujaaz, Inc. 


Caren Namalenya, Shujaaz Inc 
Paul Gitonga, Shujaaz Inc 


e Collect real-life stories about young people's experience with 
COVID-19 and vaccination. 


e The formative study informed design of the Shujaaz digital 


Norah Kopi, Shujaaz Inc media Social and Behavior Change campaign. 

Linet Juma, Shujaaz Inc e The performance and the early impact of the campaign were 
Sylvia Thuku, Shujaaz Inc tracked via a combination of the following activities: 
Christine Njeri, Shujaaz Inc e Panel SMS survey with followers of four Facebook pages, 
Farida Nzilani, Shujaaz Inc where the campaign took place: DJ Boyie and Maria Kim in 


, ` i Kenya and DJ Timamu and Pendo in Tanzania. Four questions 
Kevin Maraguri, Shujaaz Inc out of 10 in the SMS survey were the same as approved 
Rabii Kinyogoli, Shujaaz Inc BrandLift Survey questions for COVID-19 surveys. Baseline 
survey was conducted in December 2021, and endline survey 


Aiman Musa, cules Ine was conducted in February 2022. 


Winnie Nyato, Independent Consultant : 
y p e BrandLift Survey was run by Upswell team from December 


Julieth Kweka, Independent Consultant 2021 through to the end of February 2022. 

Leyla Mchawe, Independent Consultant e Natural Language Processing (NLP) and Machine Learning 

Dr. Anastasia Mirzoyants, Mirzoyants Management Consulting (ML) were used to track and analyze shifts in sentiment related 
Dr. Matti Pohjonen, Oy Agrofor Consulting to COVID-19 vaccination. 

Drew Bernard, Upswell e Traditional Facebook metrics for reach, engagement and 


deep engagement were also collected and assessed by the 
Shujaaz-Media Team. 


e Wrap-up Focus Group Discussion with members of Facebook 


Young people aged 15-24 in each country, who had access to the audience on their overall experiences with the campaign, 
internet and were using Facebook and/or Instagram tactics that did and did not work, overall learnings. 


Kenya and Tanzania, nationwide in each country 


At the beginning of the project, Shujaaz in-house Knowledge and 
Learning team conducted a formative study to: 


e Understand key barriers and motivators in young people's 
decision about vaccination. 


e Segment young people by knowledge, attitudes and practices 


related to vaccination. BG © 
j © 


Identify and describe positive outliers - young people who got 


PROJECT SUMMARY 


We are Shujaaz Inc, a network of social ventures based in Kenya and Tanzania. We believe this generation of young 
people can transform our global society for the better — that’s why we work to break down barriers so that young 
people can take control of the future. 


Our multi-media youth platform Shujaaz, is a unique digital and real-world community that gives young people the 
self-belief to succeed; through a dedicated campaign we are connecting our 4.4 million social media followers with 
the information, insights and confidence they need to champion community uptake of the coronavirus vaccine at 


a national scale. Shujaaz media campaign, conducted as an action-research, used two persuasive strategies in 
parallel to understand key barriers and motivators in young people's decision about vaccination: 
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e Shujaaz “in-house” campaign using core Shujaaz 
characters Facebook-pages: DJ Boyie and Maria 


Kim (Kenya) and DJ Timamu and Pendo (Tanzania); 


amplifying ‘survivor stories’, the characters getting 
vaccinated, sharing FACTS on the virus & vaccines 
and creating a feedback-loop for fans reporting 
their getting vaccinated. 


e Micro-influencer campaign aimed at engaging social 
media personas popular in Kenya and Tanzania. 


KEY FINDINGS 
The overall achievements of the campaign are: 


e 2M+ Shujaaz audience across East Africa of the 
#LindaFam and #Wachanjo social media campaigns. 


e Shujaaz campaign inspired a dynamic social media 
conversation on vaccination — e.g., in the average 
comments generated per post-DJ Tee 137; Maria 
Kim 105; DJ Boyie 45; and Pendo 26. 


e Even in this short campaign, Shujaaz created a 
positive shift in sentiment towards vaccination; albeit 
less pronounced in Tanzania where the national 
COVID-19 narrative is more negative than in Kenya. 


e The campaign inspired young people to get vaccinat- 
ed — the main proof of the power of #Wachanjo was 
delivered during the February vaccination drive, when 
in one day 246 people showed up and got vaccinated. 


e Young people reinforced their role as change 
agents in their communities. A key campaign goal 
was to persuade young people in the Shujaaz 
audience to talk to peers and family about the 
importance of vaccination. Across the two coun- 
tries, almost 9 in 10 young people did that, and of 
those, 8 in 10 reported that people, whom they told 
about vaccination, got vaccinated as well. 


KEY PROCESS INSIGHTS 


The use of exact, relevant words resulted in more 
focused thematic conversations — the posts and 
comments that contained vaccine words received 
significantly more engagement. It appears important to 
be direct in conversations about vaccination. 


People react differently to fictional stories and real-life 
testimonies — people are supportive and empathetic, 
when it comes to the testimonies, if they do not give 


Shujaaz Media (the largest youth 
brand in East Africa used by more 
than 9million people) designed and 
implemented a Social and Behavior 
Change media campaign in Kenya and 
Tanzania to motivate young people 

to discuss, take up, and promote the 
COVID-19 vaccination. The campaign 
inspired a dynamic Facebook 
conversation on COVID-19 vaccination 
among 2million+ young East Africans, 
which resulted in high useage of 
#LindaFam or #Waichanjo campaign 
tags; increased confidence in and 
positive sentiment towards COVID-19 
vaccination; as well as youth getting 
and endorsing the vaccine. 


themselves permission to say how they truly feel about 
the topic because of fear of appearing insensitive/of- 
fensive. When reacting to fictional stories, Shujaaz fans 
were honestly discussing their attitudes and beliefs 
towards vaccination. A campaign on a sensitive topic 
should have a mix of real-life and fictional stories. 


People use different languages to express different 
sentiment-In both countries, comments in Swahili 
were significantly more negative than those in English. 
Especially in the Tanzanian pages the differences are 
quite high between posts that use Swahili or English 
together with vaccine words. 


People with different sentiments towards vaccination 
appear online at different times of the day — There are 
a number of interesting patterns related to the days/ 
times when people are active and the types of com- 
ments they post during specific time frames, e.g., time 
of the day (Tanzania) or day of the week (Kenya). 


Not all micro-influencers were helpful in running the 
campaign — some micro-influencers were able to truly 
inspire and guide their followers, while others posted 
content that was only remotely relevant to the Shujaaz 
campaign. It is important to align with the micro-influ- 
encers beforehand on the expectations. 


RECOMMENDATIONS 
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Work with young people and get their buy-in 
because they can be powerful change agents in 
their communities. 


Ensure young people have ownership of the 
program and are included as designers/developers/ 
promoters, ensure they have an active participatory 
and leadership role not passive/recipient roles 


Ensure full transparency of how the outcomes 
of the work are going to be used to benefit other 
young people. 


Use exact positive words in communication on 
vaccine. Not instructive but using “survivor” and 
“hero” narratives and personal stories, focused on 
COVID-19 to avoid confusion. 


Use a combination of facts, personal narratives, 
micro-influencers, and fiction to talk about norms 
and behaviors associated with vaccination. 


Be careful about how you use native languages 
(e.g., Swahili or mother-tongues) in this communi- 
cation to ensure that the use of a specific language 
does not reinforce stereotypes, myths, and miscon- 
ceptions discussed by the users of the language. 


Be mindful of the larger context/narrative affecting 
the target audience — e.g., because of the specifics 
of how COVID-19 was/is discussed in Tanzania, it 
was more difficult to bring up and openly discuss 
the issue of vaccination there. 


If you are working with micro-influencers or other 
activists, who are not part of your team, make sure 
to train them and align with them on the type of the 
content and standards expected. 
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“I have been reading Shujaaz since 
chapter 96. That's the first copy | ever 
came across when | was in Embu. It 
was around 2010. | love Maria Kim's 
story and | ended up following her on 
Facebook. So, | saw her post about 
vaccination and followed peoples’ 
comments. The posts were engaging 
but people's comments were mostly 
negative, and people did not want to 
get the covid vaccine. They were saying 
it is like going for other diseases. But 

| did not let that stop me from taking 

it because | travel a lot and there was 
this time, they said you could not travel 
without the vaccine certificate. That 

is when I went to Oloitoktok hospital 
and took the Johnson's Vaccine. | did 
not find a long queue like a while back 
when people were going and finding the 
vaccines are over. The only side effect 

| got was a headache but when | slept 
and relaxed the next day it was gone. 
My friends were laughing at me that 

| would die very fast and leave them 
enjoying life. That scared me a little. | 
later overcame the fear three months 
after since | was not experiencing 
anything out of the ordinary. My life and 
health were normal.’ 


-Paul Kirimi, 22-year old from Kajiado County, Kenya 


ORGANIZATION NAME 
The Behavioural Insights Team 


GEOGRAPHY 


Chaco province, Argentina 


KEY PARTNERS 


Government of Argentina Behavioural Sciences 
and Public Policy Unit 


Chaco Province Ministry of Public Health 


FOCUS POPULATION 


Adults age 18+ who have received at least one dose of a 
COVID-19 vaccine, but who have not completed their full course. 


METHODOLOGY TYPE 


Formative phase: Mixed methods exploratory research (data ®© 
analysis, semi-structured focus groups and interviews) 


Impact evaluation: Quantitative research (randomized control trial) 


We have developed and are evaluating the impact of a WhatsApp chatbot that aims to 
increase COVID-19 vaccination rates in Chaco, Argentina. The message content and 
functionalities offered through the chat are informed by our fieldwork and the behavioural 
science literature. 


The chatbot aims to address the behavioural barriers to vaccination, including the friction 
costs of figuring out how to get vaccinated, and prompt people to act on their intentions to 
get vaccinated. It does so by (1) sending personalized messages informing users of their 
eligibility for the next dose of the vaccine, (2) providing practical information on vaccine 
centres based on the user's location, (3) prompting users to plan when and where they 

will get their next shot and (4) sending a personalized reminder the day before the user's 
chosen date to visit the health centre. 


This tool will be evaluated through a randomized control trial - the gold standard in impact 
evaluations - with vaccination outcomes measured at the individual level. 


According to Argentina’s Federal Ministry of Health, around 83% of people have received 
their first COVID-19 vaccine in the province of Chaco. Despite high vaccine acceptance, 
only 69.6% of people had received all their primary doses and less than 35% had received 
a booster. According to our fieldwork, supply does not seem to be the challenge. Instead, 
we identified some behavioural barriers that might affect uptake: 


SOCIAL MEDIA PLATFORM/S USED 


Can a WhatsApp 
chatbot 
informed by 
the behavioural 
science 
literature 
increase 
COVID-19 
vaccination 
rates? An 

RCT currently 
in progress 

in Chaco, 
Argentina. 
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e Perception of risks from COVID-19 has changed 
throughout the pandemic; our field research 
indicated that risk perception was low among our 
target population, as infection rates were very low 
in the province at the time. Pro-vaccine messaging 
that was previously shown to be effective may be 
less impactful at this stage of the pandemic, e.g. 
prosocial messages (‘getting the vaccine helps 
protect others’). 


e People face friction costs i.e., small details that 
can make vaccination appear more effortful and 
may impact the likelihood of getting a shot. While 
the Ministry of Health in Chaco assigned individual 
appointments and strategically located pop-up 
vaccination stands for first-doses, for subsequent 
shots people must actively seek information, find a 
convenient health centre, and decide when to go. 


e — Intention-behaviour gaps research shows that 
people are good at visualizing their goals, but not 
the steps it takes to achieve them. Our field research 
showed there are people who got their first doses, 
and are willing to get a third one, but have repeatedly 
postponed the task of visiting their health centre. 


KEY PROCESS INSIGHTS 
Insights from working with local partners 


e The commitment of our partners at the Chaco 
Ministry of Health and the Behavioural Science and 
Public Policy Unit of Argentina has been fundamen- 
tal for this project. Building trusting relationships and 
encouraging stakeholders’ ownership over a project 
early on is crucial for successful collaboration. 


e Given the quickly changing nature of the pandemic, 
modifications to local COVID-19 vaccination 
protocols are to be expected and affect intervention 
design. Staying informed about planned changes 
and allowing for flexibility in the intervention design 
is key to success. 


Insights from field research 


e People's health-related perceptions and behaviours 
change rapidly as the pandemic evolves. Research 
and intervention design must be updated to the 
changing circumstances. Using non-standard 
qualitative research techniques (e.g., reviewing 
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comment sections of vaccine clinic Google listings) 
can generate rapid insights, which can later be 
validated through in-person field research activities. 


The importance of collaborative solution design 
should not be underestimated. Co-designing with 
local partners and citizens through joint working 
sessions, focus groups, and user testing, led to 
significant improvements in the chatbot design. 


Insights from designing a WhatsApp chatbot 


Using WhatsApp to send official communications 
from a Ministry of Health comes with the challenge 
of creating trust in the service. Fear of scams and 
distrust in digital communication pose implementa- 
tion risks that should be mitigated. 


Developing and hosting a WhatsApp chatbot using 
government IT systems may present technical 
challenges and requires local partners to be dedicat- 
ed to the success of the project. 


RECOMMENDATIONS 


Our research is currently underway. Recommendations 
will be made based on rigorous evaluation findings, 
available in Q3 2022. 


A FOCUS GROUP TESTING THE CHATBOT 


Co-designing with local partners and citizens through 
joint working sessions, focus groups, and user testing, 
led to significant improvements in the chatbot design in 
Argentina. 


ORGANIZATION NAME GEOGRAPHY 
The Indiana University Observatory on Social Media 


KEY PARTNERS 


New Georgia Project; Reality Team 


FOCUS POPULATION 


Urban Black American Twitter Users, aged 18-40 


METHODOLOGY TYPE 
Experiment and social media intervention SOCIAL MEDIA PLATFORM/S USED 


y 


We intend to raise vaccine confidence by generating an online intervention among young Our study 
Black Americans. We will first test messages and messaging strategies by surveying our points to the 
study group. Then, we will try to create a viral process of the best messages on Twitter, potential 
using advanced network theory. Finally, we will assess the impact of our intervention on and need for 
vaccine uptake statistics. grassroots 
online 
campaigns to 
We used an experiment to understand which types of positive messages about reach young 
vaccines have the greatest potential to be spread by our participants. We found that urban Black 
our participants would be most likely to share messages that have content related Americans. 


to personal experiences of ordinary people about vaccines. Interestingly, the least 
shareable messages contained images designed to provoke fear of the consequences of 
COVID-19 infection. We also tested neutral messages with simple pictures of vaccines, 
and interactive messages that were designed to stimulate discussion about vaccines. A 
second part of our study looked at other factors regarding why people might be willing 
to share vaccine-positive messages. Unsurprisingly, we found that individuals with 
higher levels of vaccine confidence are more likely to share our messages. However, 
when we account for vaccine confidence, we also found that people with more open- 
minded thinking styles and more confidence in social media are more likely to share our 
messages. We also studied focus group data, provided by our partner, The New George 
Project, to try to understand how best to design campaigns to promote vaccines among 
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Black Americans. We found three factors related to wW VACCINE PROMOTION ON TWITTER 

low vaccination rates among Black people: (1) concern Examples of images tweeted from the campaign. 
about safety and efficacy of the shot; (2) distrust of the 

public health system, coupled with; (3) fatigue from 

exposure to its campaigns. 


KEY PROCESS INSIGHTS 
LONG COVID 


An important part of our process was working with 
members of the Black community. We received many 
invaluable insights, guidance, and data from our project 
partners. This enabled us to understand key reasons 
for vaccine hesitancy among the community and tailor 
our work to address these. We also received valuable 
guidance on ways to present grass-roots campaigns to 
members of the community. It is important to build a 
trustworthy online identity for an organization releasing 
information to members of the community, but to 
refrain from falsely assuming a Black identity for the 
site through its graphic design. Using this approach we 
developed the identity of our Twitter page for partici- 
pants in the project - https://twitter.com/bandtproject. 
The Twitter account now has approximately 1,500 
followers and is still growing quickly. 


ATT LOS CRYVOUT 


RECOMMENDATIONS 


A key reason for vaccine hesitancy within Black 
communities is the distrust of medical advice from 
the government and the public health system. Health 
campaigns in the future should make it a high priority to UNKT Pa 7 da VA 
address this issue. Our results indicate AA # W hy IGol \ AX X E D 
campaigns on social media have good potential for 
reaching many Black Americans. Those campaigns 
should target social networks in which Black Americans 
have a strong level of trust. We note that while Twitter 
is currently trusted, this may change over the next year 
due to its change of ownership. 


We have a number of recommendations about the 
types of content which would work best in grassroots 
campaigns. 


1. We found that campaigns that are personally relat- 
able work best. For example, those that use images of 
people and tell personal stories. 


2. Avoid campaigns intended to leverage fear respons- 
es. For instance, images that contain portrayals of ill or 
dead people. 
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ORGANIZATION NAME 


Unity Consortium 


GEOGRAPHY 


National 


FOCUS POPULATION 


Parents of adolescents and young adults 


METHODOLOGY TYPE 


Qualitative Research 


SOCIAL MEDIA PLATFORM/S USED 


Online boards were used for phase 2 of research (requesting 
participants to share posts or content created) 


=] 


Unity will use novel research techniques to understand the elements necessary to 
engage and educate parents through social media about adolescent and young adult 
(AYA) vaccination, ensuring AYA protection from vaccine-preventable diseases. This 


will inform a Unity-driven social media capacity-building intervention to amplify parents’ 


voices and increase vaccine confidence amongst their peers. 


Parents are wary about sharing vaccine information on social media, with a high fear 
of backlash. Additionally, any vaccine posts feel affiliated with the COVID-19 vaccine, 
which is seen as a taboo topic. Parents are more willing to share posts from sources 
they consider unbiased, in small, familiar social media groups, or anonymously. It is 
preferred that the post not be associated with political or financially benefiting sources 
(e.g., for-profit companies) and possess a suggestive/helpful tone vs. a commanding 
one. Source credibility also matters. Parents take healthcare provider's opinions into 
consideration the most, however a small minority of parents question the integrity of 
government affiliated organizations such as the CDC, and a small number of vocal 
parents find no sources to be truly credible. Most of these parents live in rural areas. 
Parents are more likely to share communications that are relevant to them and their 
peers, families, and friends, for example, simple infographics that educate on vaccine 
recommendations by adolescent age groups. Furthermore, quick reminders and 


Parents are very 
hesitant to share 
vaccine-related 
information on 
social media, 
however, they 
are more willing 
to share in 
closed, like- 
minded online 
communities 
and if the 
vaccine 
information is 
simple, relevant, 
straightforward, 
and not 
directive. 
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messages about overall health should be interwoven 
into messages about vaccines. 


KEY PROCESS INSIGHTS 


Most parents of adolescents and young adults recog- 

nize the importance of vaccines for AYAs. However, 
vaccine safety concerns are on the rise, with parents 
acknowledging the impact of social media on their beliefs. 
Furthermore, we found that parents seek information from 
sources they believe to be trusted, with 2/3 of parents 
seeking information from their doctor or healthcare 
provider. Parents are very hesitant to share vaccine-related 
information on social media, however, they are more 
willing to share in closed, like-minded online communities 
and vaccine information that is simple, relevant, and not 
directive. Finally, parents of adolescents are more willing 
to share engaging graphics and simple messages shared 
from trusted sources. As such, access to healthcare 
provider/health organization created and shared vaccine 
information and graphics is an important enabler of 
parent-to-parent online vaccine messaging. 


Credibility Continuum 


More Credibility 


E ic 


Health Care Provider 


+ Trust in personal doctors has decreased, 
though remains most important influence 


Personal Experiences 
+ Parents trust other parents and seek 
Stories and experiences 


jo 


None of the above 


+ Small but vocal group do not find any credible 
Sources, distrust government and almost all in 


rural areas Less Credibility 


Tia 


Parents are more willing to share: 

In smaller, like-minded groups; moderator present 

If advocating self-education or community well-being 
Content that is not polarizing; resonates emotionally 
If anonymous 


* Majority believe gov't agencies are 


RECOMMENDATIONS 


We recommend stakeholders: 


Focus communications on the historical success of 
vaccines 


Empower parents by acknowledging and addressing 
the reasons they are vaccine hesitant 


Provide communications that emphasize peace of 
mind to help parents feel more in control by counter- 
ing misinformation while remaining unbiased 


Include hyperlinks to increase credibility and allow 
parents to control how much information they take 
in without being overwhelmed 


Prepare parents for potential backlash online 
through education 


Provide anonymity when necessary to boost likeli- 
hood of sharing publicly 


Provide data that shows positives and negatives, 
while still being supportive of vaccines 


Make communications relevant by breaking down 
information by age, region, or vaccine feelings 


Quick reminders and messages about overall health 
should be interwoven into messages about vaccines 


CREATING TRUST 
AND CREDIBILITY 


These tables 
highlight the key 
factors in found to 

be most effective 

in encouraging 
vaccine supportive 
parents to share their 
experiences via social 
media. 


Scientific Community 


Many find comfort in scientific, 
medically backed data, especially with 
source links 


Government Agencies 


vaccine experts, interest is public health; 
while a small minority question the 
integrity of CDC, FDA, etc, 


Less Willing 
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ORGANIZATION NAME METHODOLOGY TYPE 


University of Antwerp In this study, several multi-disciplinarian methods were combined 
to gain insight from different perspectives and factors influencing 
vaccine uptake in pregnant and lactating women. 


FOCUS POPULATION A scoping review of literature was written to determine specific 
Pregnant and lactating women and women of childbearing age gaps according to our research question. 


To detect vaccine hesitancy during pregnancy on social media 
messages, Dutch data from Twitter and English data from Twitter 
SOCIAL MEDIA PLATFORM/S USED and Reddit were collected. Adapted and fine-tuned Bidirectional 


Encoder Representations from Transformers (BERT) models were 
VW Fi Reddit used to identify, and establish reasons for, vaccine hesitancy. 


To describe factors influencing vaccine uptake in the target group, 

desk research and a limited number of semi-structured interviews 
GEOGRAPHY were performed to gather a broad overview. Loop diagrams were 
Europe used to bring the variables together in a system map, creating an 

overview of the inter-linkages between the different variables. 


To evaluate potential communication strategies about maternal 
vaccination, the effect of narrative strategies to debunk mis- and 
disinformation were tested via an online Prolific experiment in 
women in the target group. The effectiveness of a pregnancy-fo- 
cused vaccine-safety label on social media channels was tested 
via a co-creation session with women in the target group. 


To detect differences across borders, a survey questioning 
attitudes towards maternal vaccination was performed online in 11 
European countries. Per country, 25 participants were included. All 
participants were pregnant or gave birth in the last two years. 


Since pregnant women, fetuses, and neonates are vulnerable to infectious diseases related morbidity and 
mortality, it is important to understand barriers in the uptake of maternal vaccines to decrease vaccine hesitan- 
cy and increase vaccination coverage in fertile, pregnant, and lactating women. Therefore, different aspects and 
interventions of social media communication and its impact on vaccine confidence, acceptance, and coverage 
in fertile, pregnant, and lactating women will be investigated. 


The results are preliminary since not all the study parts are closed. 


The main findings of this study are that approximately 40% of messages about COVID-19 vaccinations and 
pregnancy found on the investigated social platforms are hesitant towards vaccines. According to literature and 
our field work, the main concern about getting vaccinated during pregnancy (or shortly after/before) is whether 
the vaccine is safe. Concretely, people worry about whether the vaccine damages the mother and/or child, 
distorts the menstruation cycle or lactation, alters the child's DNA, or causes disease or even death. Vaccine- 
skeptics also believe that the development period of the COVID-19 vaccines was too short, and that there is a 
lack of tests that investigate the long-term effects on pregnant women and their children. 


The results of the Prolific experiment, the co-creation session testing the effectiveness of a pregnancy-vaccine- 
safety label for social media, and the results of the survey are being analyzed. 
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KEY PROCESS INSIGHTS 


The taboo of maternal vaccination is still vividly present. 
After reviewing literature and conducting research in 
Belgium and ten other European countries, we see that 
overall women are more hesitant about vaccination during 
pregnancy and lactation than vaccination in general. 


With the COVID-19 pandemic, the word ‘vaccine’ was 

all over the internet. Social media does have a certain 
role in providing information and sharing opinions 
about vaccines. By screening Twitter and Reddit 

during the first wave of the pandemic, we found that 
approximately 40% of the messages about vaccination 
and pregnancy were hesitant towards vaccination. The 
most important reason to doubt maternal COVID-19 
vaccination was the concern about safety for the fetus/ 
baby, fed by the fact that pregnant and lactating women 
were initially excluded from clinical trials. 


We identified other factors for unwillingness to get 
vaccinated during pregnancy or lactation by performing 
a debunking study in our target group. Faith in intuition, 
pre-vax attitude, and distrust in health system are 
factors that are significantly and negatively related to 
vaccine confidence. Also, participants with religious 
affiliation were more likely to be skeptical towards a 
vaccine and to believe in vaccine conspiracy theories. 


SURVEY ABOUT MATERNAL 
VACCINE CONFIDENCE IN 
EUROPEAN COUNTRIES 


To compare our Belgian 90 
findings with the vaccine 
attitude of pregnant and 


lactating women in other 70 
countries, we set up a survey 
distributed in ten other 50 


European countries. The overall 
findings show a wide variety in 
vaccine confidence between 40 
the countries. Additionally, 
the confidence in maternal 3 
vaccines was presented 
lower than in vaccines in 


effectiveness, and compatible o 


with religious beliefs. aka 


m important 
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Bulgaria 


Social media channels should be aware of the 

power they have to balance the way information 
reaches the individuals. By creating an effective 
misinformation label, we want to positively contribute 
to vaccine confidence via social media. These sorts 

of interventions are necessary to improve vaccine 
confidence in pregnant and lactating women, a hard-to- 
reach group, worldwide. 


RECOMMENDATIONS 


This study indicates the importance of the provision 

of correct information and clear communication about 
vaccines. Medical and governmental organizations 
together with physicians, pharmacists, and other 
healthcare professionals play an essential role in these 
matters. Health care providers should give the right 
information about maternal vaccination and should 
encourage pregnant and lactating women to critically 
read social media messages. By showing the way to 
scientifically-based websites and to reliable sources, 
women should be able to make a well-informed choice 
about vaccination during pregnancy or lactation. 


Furthermore, including pregnant and lactating women 
in pre-marketing clinical trials should be encouraged, 
since lack of testing on this target group is one of the 
reasons why women doubt or refuse to get vaccinated 
during pregnancy or lactation. 


Confidence on vaccines in pregnancy: by country 


Finland France Germany Italy Netherlands Poand Slovenia 


mSefe mffetive sm Compatible withmyreigious beet 


KEY PARTNERS 


University of California, San Francisco 


GEOGRAPHY 
Haryana, India 


Post graduate Institute for Medical Education and Research, 
Chandigarh (PGIMER) 


Survival of Women and Children Foundation (SWACH) 
Indraprastha Institute of Information Technology, Delhi (IIIT, D) 


FOCUS POPULATION 


Pregnant and breastfeeding women 


SOCIAL MEDIA PLATFORM/S USED 


METHODOLOGY TYPE 


Formative surveys and qualitative interviews; community surveys 
implemented before and after intervention 


Chatbots 


> p 


Expanding high quality vaccine education is critical for pregnant women, yet uptake in India 
is low due to fears and misconceptions. Bots offer an innovative opportunity for automating 
responsive and efficient education. Using a human-centered design approach we will develop 
and test a behavior change communication Bot for COVID-19 vaccination education in 
collaboration with Indraprastha Institute for Information Technology, Delhi; the Post-Graduate 
Institute for Medical Education and Research and the Survival for Women and Children 
Foundation. We will test the impact and acceptability of Bot deployment on vaccine hesitan- 
cy and acceptance among pregnant women. 


COVID-19 vaccination remains low among pregnant and breastfeeding women in northern 
India; however, most want to be vaccinated. Our team developed a chatbot to efficiently 
address COVID-19 vaccination concerns, which we deployed over existing WhatsApp groups 
for pregnant, postpartum and parenting women. Formative research revealed barriers to 
COVID-19 vaccination include fetal and infant safety concerns, health provider refusal, and 
gender norms. The chatbot addressed key misconceptions and concerns and provided 
evidence-based guidance. Promoting chatbot engagement required time and human resourc- 
es. During initial deployment, research team members answered questions regarding chatbot 
technology and function, and users shared experiences. Group members shared the chatbot 
with family and friends, which increased engagement. The chatbot was easy to engage with 
(78.8%), provided easy-to-understand information (91.4%) and met all information needs 
(96.7%). 88.9% reported the chatbot improved their COVID-19 knowledge a lot, 71.7% recom- 
mended it to others and 88.3% shared information from the chatbot with others. No negative 
feedback was received, and few technical issues occurred. 


Chatbots in 
WhatsApp 
groups are 
feasible, 
acceptable 
and effectively 
address 
COVID-19 
vaccination 
concerns 
among 
pregnant and 
breastfeeding 
women in rural 
North India 
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During intervention effectiveness assessment, most 
community survey respondents had received at least 
one COVID-19 vaccination dose (86.2% pre-intervention 
vs. 87.7% post-intervention). Full vaccination (2 doses) 
increased from 54.6% pre-intervention to 61.1% 
post-intervention. Increases observed in vaccination 
from pre to post-intervention were marginal for any 
vaccination (OR 1.15, 95% Cl 0.99-1.36, p=0.06) but 
met the threshold for statistical significance for full 
vaccination (OR 1.21, 95% Cl 1.09-134, p<0.001). No 
differences were identified in level of vaccine hesitancy 
from pre- to post-survey; few unvaccinated study 
participants reported not having tried to access COVID 
vaccination. Major vaccination barriers included being 
considered ineligible for vaccination due to pregnancy 
or breastfeeding status (70.7% and 63.5%, pre and 
post- intervention respectively). Major reasons for 
non-vaccination included being pregnant or breastfeed- 
ing, with fewer reporting family or health care providers 
as barriers. Many unvaccinated participants were very 
likely to get COVID-19 vaccination if offered (52.4% and 
58.5%, respectively); few would not consider vaccina- 
tion (12.2% and 17.0%, respectively). 


KEY PROCESS INSIGHTS 


Developing a chatbot to address vaccine hesitancy 
among pregnant and breastfeeding women in northern 
India was straightforward, particularly given the early 
stage of our project, and inclusion of a formative re- 
search component to identify key barriers and concerns 
to address. However, chatbot deployment was more 
complicated. We learned that it takes time and human 
resources to constantly motivate potential participants 
to engage with the chatbot. The lack of familiarity with 


86.1% |] 87.7% 79.2% A 
| Post 


Pregnant 


Total Population 


OR 1.31, 95% CI 1.12-1.53, p 


OR 1.16, 95% CI 0.99-1.36, p=0.064 0.001 
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chatbot technology among our study participants 
resulted in some queries and concerns, and our team 
plus early adopters helped allay any fears. Our project's 
situation within an existing health education structure 
was a benefit to this study, given the trusted informa- 
tion source, and understanding how to best scale this 
technology will involve considering how to capitalize on 
networks and trusted health information structures. 


RECOMMENDATIONS 


Chatbots, deployed over existing WhatsApp groups, 
were highly acceptable and feasible amongst a popu- 
lation of women who often face barriers to obtaining 
information due to gender norms restricting women's 
mobility outside the home. Participants requested 
more chatbots on other topics—suggesting that future 
implementors or program and policy makers could use 
this approach to provide information about a variety of 
health topics to this population. We recommend that 
chatbots such as this be expanded and that future 
researchers test different ways of reaching people with 
chatbots, outside of existing WhatsApp groups, such as 
Facebook or other social media outlets. 


We also found that Chatbots held potential for social 
network effects, where participants could share with 
their family and friends. Thus, we recommend that this 
type of tool could be used to help empower women to 
share information with their families, who may ultimate- 
ly be the decision-makers in this context. Providing 
information to family members, who are often not 
targeted with health information, is key, and chatbots 
that were provided by trusted sources of health infor- 
mation allowed family members to access information 
in a way they might not otherwise be able to. 


85.4% 


84.5% 


CHATBOT 
IMPACT 


This shows 

the COVID 
vaccination rate 
by pregnancy/ 
postpartum 
status, 
comparing pre 
to post chatbot 
intervention. 


<4 


Postpartum 


OR 1.08, 95% CI 0.80-1.46, p=0.604 


v COVID VACCINATION, VACCINATION INTENT, AND HESITANCY AMONG PREGNANT AND POSTPARTUM 
WOMEN, HARYANA, INDIA 


Total Population 


Pre-intervention Post-intervention 
n=609 n=434 
n bi} n F 
COVID Vaccination 
Vaccinated against COVID-19 
No E 13.5% 53 13.3% 
Yee, 1 dose 194 32.1% 116 270% 
Yes, 2 doses 327 54.0% 261 60.7% 
Months since latest vaccination" 
Median, OR 2.0 {10-4.0} 2.0 (1.0-4.0) 
Min, Max 0-10 -12 
Vaccination Intent 
Unsuccessfully tried to get vaccinated 
No 11 13.4% T? 13.5% 
Yes, told could not be vaccinated 
because of pregnancy or 
breastfeeding 58 70.7% 33 63.5% 
Yes, told could not be vaccinated for 
another reason 13 15.3% 12 23.1% 
Likelihood of getting COVID-L9 vaccine 
if offered at mo cost’ 
Vary likely 43 52.4% 31 545% 
Somewhat likely 20 24.4% 7 13.2% 
lam mot sure 7 E5% 5 3.4% 
Somewhat unlikely 1 1.2% o o.0%4 
Very unlikely 1 1.2% i 1.3% 
No, but would consider for future 10 12.2% g 17.0% 
Vaccine hesitancy 
Reason for non-vaccination! 
Waiting until no longer pregnant 28 34.1% 20 37.7% 
Lam waiting untl | am mot 
breastieeding 32 39.0% 21 39.65 
Health care worker said | could mot ā 7.3% 3 5.7% 
Family said | could not a 0.8% 3 5.7% 
lam scared 2 2.4% Fi 38% 
Other 6 7.3% 4 75% 
Vaccine communication 
Recent communication with 
friends family re OOVED-19 vaccine 
No 156 30.8% 165 39.2% 
Yer 359 59.4% 230 53.4% 
Not sure 59 38% 32 7.4% 
af yoccnated 1-2 doses, "i 
unveccinated 
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ORGANIZATION NAME 


University of Chicago 


GEOGRAPHY 
Kenya and Nigeria 
Molly Offer-Westort, Assistant Professor, University of Chicago 


Leah Rosenzweig, Director, Development Innovation Lab, 
University of Chicago 


KEY PARTNERS 
Busara Center for Behavioral Economics 


FOCUS POPULATION 
Facebook users 


SOCIAL MEDIA PLATFORM/S USED 


METHODOLOGY TYPE 


G Facebook Messenger 


Stage 1: multi-arm adaptive experiment 


Stage 2: standard balanced randomized control trial 


During mass vaccination campaigns, social media platforms can facilitate broader 
access to public health information, but they may also engender vaccine hesitancy 
through the spread of false or misleading information. We deploy a Facebook 
Messenger chatbot to test interventions designed to target sources of vaccine 
hesitancy among social media users in Kenya and Nigeria. 


We set out to determine whether using a chatbot to deliver targeted messages 

to people who are hesitant to get vaccinated could improve COVID-19 vaccine 
acceptance in Kenya and Nigeria. Prior to initiation of our quantitative research, 

we conducted focus group discussions to understand relevant concerns about 

the vaccine and possible messages to address these concerns. We then created a 
Facebook page for our study, and recruited Facebook users in these countries who 
had not yet been vaccinated to engage with our page on Messenger. Responses 
from our page were delivered by a chatbot. In the first stage of the study, we asked 
respondents about their concerns related to the vaccine and used an adaptive 
algorithm to learn which messages were most effective at addressing specific 
concerns. In the second stage, we compared the concern-responding version of the 
chatbot, with optimized messages, to a public service announcement condition and 
a control condition. Users who engaged with the concern-responding chatbot were 
more likely to report willingness to get the vaccine if it were available today, and 
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Engaging with a 
concern-eliciting 
chatbot leads 
Facebook users in 
Kenya and Nigeria 
to be more likely to 
report willingness 
to get the vaccine 
if it were available 
today, and more 
likely to report 
higher levels 

of enthusiasm 
about getting 
vaccinated, as 
compared to users 
in a public service 
announcement 
chatbot or control 
condition. 


more likely to report higher levels of enthusiasm about 
getting vaccinated, as compared to the public service 
announcement condition or pure control condition. 
These differences are large and highly statistically 
significant. The PSA condition also has higher levels 
than the pure control condition on these metrics, 

and, when controlling for pre-treatment responses, is 
statistically distinguishable from the control. However, 
the magnitude of the difference is much smaller. 


KEY PROCESS INSIGHTS 


Focus group discussions were key to ensuring we un- 
derstood concerns of people who were hesitant about 
the vaccine in our study contexts, and that we were able 
to develop messaging relevant to those concerns. Our 
focus group discussions included respondents across 
different social/demographic groups (men, women, 
older, younger, urban, rural) in both Kenya and Nigeria. 


When we moved to our online sample, we also targeted 
our Facebook advertisements to obtain respondents 
across these groups. To do so, implementation took a 
bit longer and we spent more money on advertising to 
reach “harder to reach” populations on social media (for 
instance, older women in Kenya and Nigeria), but this 
was necessary to get a balanced and diverse sample. 
However, our respondents were, on average, younger, 
more likely to live in urban areas, and higher educated 
than the overall populations in Kenya and Nigeria. 


By advertising on Facebook, we can reach social media 
users where they are across both Kenya and Nigeria. 
To the extent that people get information and respond 
to it on social media, we are able to send messaging 
interventions related to the vaccine very close to users’ 
online behavior and media environment. However, we 
know that not all Kenyans and Nigerians are online/on 
Facebook so in terms of external validity, this study is 
limited to findings that can apply to social media users 
in these countries. 


RECOMMENDATIONS 


Our findings suggest that 1. chatbots can be effective 
channels to deliver targeted information related to 
health and policy interventions; and 2. chatbots that 
engage people and ask them questions may be more 
effective than generic, non-interactive messaging. 


Data For Society 


Have any thoughts or concerns about the 
COVID-19 vaccine? Tell us! 


It can be hard to figure out 
what information we see on 
social media is true or false. 


There has been a lot of 
misinformation spread about 
the COVID-19 vaccine. 


Experts say there are 7 types 
of misinformation. 


#1. Fabricated content 


A CONCERNS ABOUT VACCINES 
Screenshot from chatbot sharing 7 types of 
misinformation. 
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ORGANIZATION NAME 
University of Chicago, Department of Economics 


GEOGRAPHY 
Freetown, Sierra Leone 


Fatu Conteh (RAMLabs), Rachel Glennerster (University 

of Chicago), Samantha Horn (Carnegie Mellon University), 
Anne Karing (University of Chicago), Tom Sesay (Expanded 
Programme for Immunization) 


KEY PARTNERS 


RAMLabs; Ministry of Health and Sanitation of Sierra Leone; 
UNICEF 


FOCUS POPULATION 
Adults, WhatsApp users, Urban 


METHODOLOGY TYPE 
Randomized Experiment, Misinformation Listening 


COVID-19 vaccination rates in Africa lag behind other continents, at 22% compared to 67% 
in the rest of the world. Vaccine supply is no longer a limiting factor in many Sub-Saharan 
African countries, however rumors and misinformation about the vaccine’s safety and 
efficacy hinder demand. Our project aims to harness the power of social media and a 
proven strategy to boost immunization: social signaling. Specifically, we offer people living 
in Freetown, Sierra Leone, the opportunity to share their vaccination status via a credible 
and personalized WhatsApp image and a colorful bracelet that shows a person is fully 
immunized against COVID-19. We hypothesize that these online and offline signaling 
interventions enable individuals to show their friends, family, colleagues, and other social 
contacts that they took the vaccine, thus providing positive counterexamples to vaccine 
misinformation. We evaluate our interventions’ impact on participants’ decisions to share 
WhatsApp messages about the COVID-19 vaccine with their social network, and vaccine 
uptake within the social network. To do so, we conducted a randomized field experiment 
in which individuals who had just taken the vaccine received either i) an offer to share 
their vaccine status on WhatsApp through a credible and personalized image (Vaccine 
Status Image), ii) intervention and a colored bracelet showing that they are vaccinated 
(Vaccine Status Image and Bracelet), or iii) an opportunity to send any message on 
WhatsApp about the vaccine to their contact (Control). We collected information to map 
participants’ WhatsApp (online) and in-person (offline) social networks, and captured 
their decision to share a message with each of their network contacts. Through a novel 
UNICEF crowdsourcing platform, we also collected data on word-of-mouth and social 
media messages about the COVID-19 vaccine from social mobilizers and social media 
listeners. This allows us to assess the spread of different rumors and misinformation in 
communities where we implemented our interventions. 
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There is 
untapped 
potential for 
leveraging 
social networks 
in vaccine 
mobilization 
campaigns: 
vaccinated 
individuals 
frequently 
interact with 
people who do 
not know about 
their vaccine 
status and are 
not vaccinated 
themselves. 


KEY FINDINGS 


Participants’ social networks are composed of family 
members (32%), friends (40%), and more distanced 
connections (23%) such as colleagues and acquaintances. 
Importantly, participants only know the vaccine status 

of 56% of their contacts, and of those, 47% are not 
vaccinated. Overall, 70% of participants share vaccine- 
related messages with their network. The main reasons 
for not sharing are time constraints, privacy concerns, and 
fear of backlash. 


The vaccine status image treatment significantly reduced 

the proportion of contacts participants shared with by 5 
percentage points compared to control; which may be 
explained by participants’ preference not to disclose/highlight 
their vaccine status. First, we see that it is the proportion of 
friends participants shared with and not the proportion of 
who shared with at least one driving the reduction in sharing. 
Secondly, we observe that 60% of messages sent in the 
control group where participants wrote their own message, 
do not disclose their vaccine status. 


Adding a bracelet to the vaccine status image counteracted 
this decline, significantly increasing the percentage of 
participants who shared by 10 percentage points compared 
to control. Because the bracelet was introduced as a lottery, 
participants experienced a positive reward in the bracelet 
group and a disappointment in the other groups. This likely 
influenced participants’ willingness to invest efforts into 
sharing, leading to more sharing with people they wouldn't 
normally share with (e.g., more socially distant contact). 
The 12 percentage points increase in sharing with spouses 
suggests that participants may have also anticipated some 
of their contacts to see their bracelet in person, and decided 
to inform them via WhatsApp beforehand. 


KEY PROCESS INSIGHTS 


Social media and communication medium in urban 
Sierra Leone 

While the most commonly used social media in urban Sierra 
Leone is WhatsApp, the population using WhatsApp is still 
somewhat limited: 37% of people we approached were 
ineligible to enter our study because they do not use the 
application. In most cases, it is because they do not have a 
smartphone or do not have access to internet on their phone. 
We piloted the same interventions with button phone users, 
and found that people we interviewed do not seem to use 
SMS to communicate with their social network. 


Working closely with vaccine teams 

A relevant insight for any other research or implementing 
organization working on vaccines is that working in 
partnership with vaccine teams at health centers and 
building good relationships with them was very important 

to build respondents’ trust and make them feel comfortable 
speaking to our field staff. Our partnership with the 
Expanded Programme for Immunization made this possible, 
and helped us design vaccine status images that were 
credible via the inclusion of the Ministry of Health's logo. 


Social network research and interventions 

Finally, we learned that people feel comfortable sharing 
information about their social networks, and recent 
conversations on WhatsApp offer a mix of different 
types of relationships people may have. This novel social 
network elicitation method has potential to improve the 
evaluation of interventions designed to directly target or 
leverage social networks. 


RECOMMENDATIONS 


Our study suggests that there is untapped potential 

for leveraging social networks in vaccine mobilization 
campaigns: vaccinated individuals frequently interact 

with family, friends, and more socially distant contacts on 
WhatsApp or in person who do not know about their vaccine 
status and are not vaccinated themselves. Governments 
and other implementers should consider ways to encourage 
vaccinated individuals to share private messages about the 
vaccine within their social network. This could be done either 
through giving them a small reward for getting vaccinated, 
creating positive association with their action, and increasing 
the likelihood that they want to tell people about it. It could 
also take the form of an information intervention that tackles 
current barriers to sharing by highlighting its positive returns 
and limited backlash. 


Restricting people to share that they got vaccinated as 
opposed to their own message reduced by 20% the extent 
to which people shared within their network. However, it 
increased the sharing of messages disclosing vaccine 
status as opposed to general information/encouragement. 
This highlights an important trade off implementers 

need to take into account when encouraging individuals 
to share any message and their vaccine status more 
specifically. Because sharing of vaccine status increases 
visibility on vaccine take-up within social networks 

and communities, we expect the former to have a 

larger influence on others’ decision to vaccinate. Data 

on vaccination outcomes among respondents’ social 
networks is still being analyzed to test this hypothesis. 
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ORGANIZATION NAME GEOGRAPHY 
University of North Carolina at Chapel Hill USA 


KEY PARTNERS 


North Carolina State University, Stanford University, University of 
California, Berkeley, National Bureau of Economic Research 


FOCUS POPULATION 


People in counties with low vaccination rates 


METHODOLOGY TYPE 


: ‘ SOCIAL MEDIA PLATFORM/S USED 
Randomized control trial 


In October 2021, Republicans were three times as likely as Democrats to be Our public 
unvaccinated against COVID-19. It turns out that a problem caused by politics also service 

has a political solution. We conducted a large scale randomized controlled trial announcement 
last fall to try to increase vaccination uptake among Republicans. We used Donald caused an 
Trump, a vaccinated Republican, as our messenger. In our 27-second public service increase in 
announcement, viewers see a news report from March 2021 that says Trump COVID-19 
received the vaccine paired with his actual endorsement of it on Maria Bartiromo's vaccinations. 


FOX News show. We ran the PSA for three weeks on YouTube, and then compared 
whether vaccinations increased in the counties that got the Trump vaccine endorse- 
ment PSA compared with those that did not. 


After winning a grant from the Vaccine Confidence Fund to buy ad time on 
YouTube, we selected at random over 1,000 low vaccination counties to receive 
the “Trump treatment” along with another set of 1,000 plus similar counties that 
would not receive it. We ran the PSA for three weeks, and then compared whether 
vaccinations increased in the counties that got the Trump vaccine endorsement 
PSA compared with those that did not. 
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They did. By alot. The average county that got the ad 
recorded an increase of 103 vaccinations more than 
the counties we did not treat. With over 1,000 counties 
getting the Trump endorsement, that translates into 
better than 100,000 more vaccines. Our ad budget was 
slightly less than $100,000 dollars, which means that 
the cost of each new vaccine was less than a dollar. 
That is far more efficient than other efforts that states 
tried to increase vaccine uptake such as lotteries. 


The key recommendation is that politics should be part 
of the solution to problems caused by politics. In the 
United States, vaccination efforts have been hindered 
by politicization and a series of anti-vaccine messages 
from prominent Republican politicians. Therefore, we 
reasoned that messages from trusted Republican 
politicians (e.g., Trump) could help persuade skeptical 
members of the public to get vaccinated. 


Those insights can be applied to future cases, although 
the specifics of the message and the messenger 


Vv VACCINES AND POLITICS 
Clip from PSA encouraging COVID-19 vaccinations. 


FOX 13 News Utah 
March 16th, 2021 


may vary. But there is no doubt that new vaccination 
challenges await. Before the pandemic, surveys found 
no partisan differences when it came to getting the flu 
vaccine or supporting childhood or adult vaccinations. 
However, a survey UNC-Chapel Hill researchers fielded 
in spring 2022 reveals that anti-COVID 19 vaccine 
sentiment among Republicans appears to be leaking 
into their opinions about other vaccines. 


We found that Democrats were consistently 10 to 15 
points more likely than Republicans to say they 1) had 
gotten the seasonal flu vaccine this year, 2) thought 
that it was important for children to get all childhood 
vaccines, and 3) tended to get new adult vaccines, such 
as for shingles and pneumonia, when they become 
available. 


Public health professionals, then, will likely have to 
grapple with how best to confront a political fault line 
in vaccine attitudes beyond COVID-19. If the success 
of the Trump vaccine endorsement is any indication, 
politics is likely to be part of the solution. 


= 


COVID-19 VACCINE 


Donald Trump is urging all Americans to 
receive the covid19 vaccine. "I would, 
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GEOGRAPHY 
US-Based Online Participants 


ORGANIZATION NAME 


University of Washington (Department of Human-Centered Design 
& Engineering) 


Gary Hsieh, Associate Professor 
Katharina Reinecke, Associate Professor 
Joshua Liao, Associate Professor 
Spencer Williams, PhD Student 


Joy Lee, Program Manager 


FOCUS POPULATION 
US-Based Online Participants 


SOCIAL MEDIA PLATFORM/S USED 


Mechanical Turk 
Prolific 
LabintheWild 


METHODOLOGY TYPE 
Co-design, interview, survey, experiment 


Scientific findings demonstrating COVID vaccine efficacy could most directly answer people's 
questions about the vaccines, boost vaccine confidence, and increase vaccination rates. 
However, these findings are difficult to communicate to audiences who may have a lack of 
trust in, and/or experience with, science. Through codesign sessions and large-scale online 
studies, we seek to develop guidelines for integrating and visualizing science-backed vaccine 
knowledge into social media posts. Our results will improve access to, understanding of, and 
confidence in vaccine research for diverse people. 


Our findings help advance our understanding of vaccine hesitancy, as well as strategies to 
help improve awareness and trust in vaccine research. 


First, our results showed that there are different types of vaccine hesitant individuals with 
different concerns and needs. We identified three broad groups: deferrers, who were waiting 
for more information before deciding to get vaccinated; unconcerned, who did not think 
COVID-19 was dangerous enough to warrant the risk/cost of vaccination; and rejecters, who 


strongly opposed the vaccine, and didn't trust institutions like the government, pharmaceutical 


companies, or researchers. Though each group had different informational needs, concerns 
about COVID-19 vaccine safety were a major issue for all these groups. 


Second, we found that communicating the risks of COVID-19 vaccine side effects using 
verbal probability descriptors (e.g., common, rare) was associated with higher, less accurate, 
and more variable perceptions of risk than using numerical probability descriptors (e.g., 4%, 
0.0035%). This could ultimately impact how people make vaccination decisions, particularly 
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Infographics 
showing the 
amount and 
consistency of 
Covid vaccine 
research 
helped improve 
awareness 

of published 
research and 
belief that 

the evidence 
shows 
vaccines are 
safe. 


for lower probability and more serious events such 

as myocarditis or cerebral venous sinus thrombosis 
(CVST). For serious complications such as myocarditis 
and CVST, we found that participants who greatly 
overestimated the risk (by three orders of magnitude) 
were more likely to report that social media was 

the strongest influence on their beliefs of COVID-19 
vaccines. 


Third, communicating the amount and consistency 

of scientific research on COVID-19 vaccines helped 
improve people's awareness of how much research 
was done, perception of research consensus, and 
agreement that research showed consistently low 
rates of adverse reaction. Furthermore, with this more 
transparent approach, people were more likely to trust 
the accuracy and honesty of the infographic, compared 
to the CDC web page. 


KEY PROCESS INSIGHTS 


We found that using co-design as a strategy to develop 
an intervention worked well. Technology-focused 
co-design stresses the importance of co-making 

(e.g., brainstorming, prototyping), instead of simply 
soliciting feedback. By framing the sessions as a way 
of building tools suited to the needs of our participants, 
our participants were very open with us about what 
information they might find useful, what might help 
them decide to get vaccinated, and the pros and cons 
of our early designs. 


We also found it was helpful to iterate on our inter- 
vention with smaller pilot studies. Originally, we used 

a quick round of pilot studies to adjust the amount of 
information to be shown, to reduce information over- 
load. Moreover, some participants seemed concerned 
about bias or manipulation of the data. We addressed 
these challenges by reducing to only the most relevant 
information, explaining how we systematically sampled 
studies, and reminding participants they are free to use 
the information however they want, which appeared to 
strengthen our intervention. 


RECOMMENDATIONS 


Our work demonstrated that with appropriately designed 
infographics, we can improve awareness of COVID-19 
scientific research, highlighting vaccine safety in a 
believable way. Although we often think more concise 
messages or research summaries may be better, our 
findings suggest that providing more transparent and 
complete information about each study—including 
studies with divergent findings—can lead to a more 
trustworthy, informative, and compelling message. 


In addition, while getting people to change their 
vaccination behaviors is hard, our findings suggest 

one positive outcome of these efforts: We found that 
providing this information also helps provide those who 
are vaccinated to use the provided information to talk 
about the vaccine within their own social network. 


Condition Vaccine hesitancy | Vaccine hesitancy | Science Science Increased PRE/POST 
due to safety (pre, due to safety credibility (pre, credibility (post, | vaccination MEASURES AND 
1-5 likert) (post, 1-5 likert) 1-5 likert) 1-5 likert) intention? (n) OUTCOMES 
CDC Baseline 2.89 (1.50) 2.79 (1.53) 3.48 (1.25) 3.53 (1.07) 22/305 Table 1: Related 
| metrics from our 
Intervention 2.76 (1.41) 2.58 (1.46) 3.65 (1.19) 3.72 (0.86) 25/289 experiment, showing 
Intervention + | 2.74 (1.45) 2.64 (1.44) 3.53 (1.30) 3.65 (0.84) 19/269 pre/post measures 


Efficacy Info 


Condition Change in perceived Change in perceived research 
quantity of research consensus (5-point likert, 
(5-point likert, 3=neutral) 3=neutral) 

CDC Baseline 2.83 (1.31) 2.89 (1.34) 

Intervention 3.18 (1.36) 3.35 (1.34) 

Intervention + | 3.08 (1.37) 3.22 (1.26) 


Efficacy Info 


of vaccine hesitancy, 
science credibility, 
and any increases in 
vaccine intention. 


Change in perceived consistency 


Table 2: Related 
metrics from our 
experiment, showing 


of low vaccine risk (5-point likert, 
3=neutral) 


| 2.91 (1.27) changes in outcome 
measures: perceived 
3.46 (1.25) research quantity, 
3.36 (1.33) consensus, and 


consistency in low 
vaccine risk. 
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ORGANIZATION NAME GEOGRAPHY 


University of Washington (Institute for Health Metrics and United States of America. The visualization tool for COVID-19 
Evaluation) vaccine hesitancy shows results at the ZIP code and county level, 
and the analyses of reasons behind hesitancy and associated 
demographic trends was conducted at the individual level, with 
KEY PARTNERS consideration for clustering within states. 


IHME, COVID Collaborative 


FOCUS POPULATION 


Residents of all counties and ZIP codes across the US who were 
respondents of The Delphi Group at Carnegie Mellon University U.S. 
COVID-19 Trends and Impact Survey, in partnership with Facebook. 


METHODOLOGY TYPE 


Surveys, social media listening, and local knowledge gathering 
SOCIAL MEDIA PLATFORM/S USED 


efi 
nyong 
kiz 


Minimizing and preventing further surges of COVID-19 is contingent on overcoming 
vaccine hesitancy. To achieve that, policy makers need to be equipped with granular 
information on what type of interventions can improve acceptance, in order to 
strategically invest resources in communities with greatest potential to increase 
vaccination rates. The Institute for Health Metrics and Evaluation, in partnership 


Our analysis of 
COVID-19 vaccine 
hesitancy in the 
United States 


with the COVID Collaborative, developed a data visualization tool that maps vaccine has continually 
hesitancy is the US at the ZIP code and county-level using survey data collected via suggested strong 
social media. geographic and 


This project sought to utilize social media to track community-level trends in hesi- demographic 
tancy and understand how and why hesitancy varies among US communities. Data heterogeneities. 
collected from surveys implemented on social media was used in combination with 

methods to monitor social media channels and gather community-level knowledge 

about reasons for hesitancy and the factors that could improve vaccine acceptance. 

Given the rapidly changing dynamics of the pandemic, our approach of providing 

regular updates on community-level vaccine hesitancy via the online visualization 

tool, paired with a social media campaign in real time has provided timely support to 

state and local policy makers. This project has demonstrated the nature of guidance 

that can be generated from bringing together a mix of methodologies including 

surveys, social media listening, and local knowledge gathering, and which could be 

applied to settings around the world. 


122 | VCF IMPACT REPORT | 


KEY FINDINGS 


Our analysis of COVID-19 vaccine hesitancy by zip code 
in the United States has continually suggested strong 
geographic heterogeneities. While overall hesitancy de- 
creased substantially over time across the majority of 
the country, there are notable persistent counties where 
hesitancy has remained high. Exploration of hesitancy 
at such a fine resolution has provided tools for policy 
makers to implement location-specific strategies for 
combatting vaccine hesitancy. 


In addition to understanding geographic differences 

in hesitancy, we aimed to quantify demographic and 
individual characteristics that were most associated 
with COVID-19 hesitancy for targeted action. We found 
that not all demographic groups were equally hesitant 
about COVID-19 vaccination. In general, older age 
groups had less vaccine hesitancy compared to 18-24 
year olds as did those with more education, compared 
to those with less than a high school degree. 


We also explored demographic differences in the cited 
reasons for vaccine hesitancy and again found stark 
heterogeneities in individual characteristics. The overall 
probability of selecting no need for the COVID-19 
vaccine among all reasons for hesitancy was 30%, 


but a Non-Hispanic White male aged 18-24 had a 46% 
probability of selecting this answer while a 35-44-year- 
old Non-Hispanic and Asian female aged 35-44, in 
contrast, only had a 12% probability. 


In contrast, a Non-Hispanic Asian female with a graduate 
degree, for example, had a 52% probability of saying they 
wanted to wait and see if the COVID-19 vaccine was safe 
before getting it versus 50% of respondents overall. 


RECOMMENDATIONS 


Our research has highlighted large disparities in 
vaccine hesitancy within the United States. Our findings 
highlight the need to understand vaccine hesitancy and 
the reasons behind it at the local level and for specific 
populations within the US. Our results provide critical 
input on the variation in vaccine hesitancy across 
several important population characteristics, including 
where people live, in addition to demographic character- 
istics such as race/ethnicity, age, and gender. For future 
vaccination efforts, it’s important to understand the 
drivers in vaccine hesitancy for each of these groups 

of the population and tailor the approaches aimed to 
improve vaccine uptake to incorporate the information 
about the level of hesitancy as well as the main reasons 
behind it. 


VACCINE HESITANCY 
LEVELS BY COUNTY 


This map highlights areas of 
the US that would benefit most 
from increased vaccination 
acceptance. This view shows, 
by county, survy respondents 
who answered “Yes, probably” 
or “No, probably not” when 
asked “If a vaccine to prevent 
COVID-19 were offered to you 
today, would you choose to get 
vaccinated?” as of February 24, 
2022. 


DATA SOURCE: The Delphi Group at Carnegie 
Mellon University, U.S. COVID-19 Trends and 
Impact Survey, in partnership with Facebook. 
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ORGANIZATION NAME GEOGRAPHY 


Upstream Thinking Rural, southern United States 


FOCUS POPULATION 


Unvaccinated African Americans 


METHODOLOGY TYPE 
Qualitative interviews 


HCD led solutioning 


Quantitative survey 


SOCIAL MEDIA PLATFORM/S USED 


The majority of strategies addressing vaccine hesitancy to date have sought to act For many, vaccine 
on the supply side of content by prefacing, restricting, or removing content. This hesitancy is a result 
project aims to affect the other side of the equation which has received much less of unmet needs 
attention - existing demand. How might we channel the existing demand to formulate in the demand 
personal opinions to shift online behavior towards vaccine confident content and for information to 
interpretations? make a decision. 
Providing 
information that 
In many ways, the behavior of the unvaccinated population is unexpected and meets these needs 


inconsistent. In spite of historical precedent of vaccination, Where they fully trust their is more effective 
doctors and uptake other immunizations, they are motivated uniquely within COVID-19 
context and common desire for safety for self and family, to arrive at their own 
conclusion, rather than defer the decision to medical and public health experts. 


than simply telling 
people what to do. 
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Our research revealed that there are 5 drivers of 
demand that currently drive their consumption for 
COVID-19 vaccine content. 


e Balance and diversity—Individuals seek out counter- 
narratives to the mainstream narrative to balance 
out the perceived mainstream bias 


e Freedom to choose (decision autonomy)- 
Individuals are strongly driven to make an informed 
decision for themselves, so they seek out content 
that informs rather than instruct 


e Trust-Individuals seek out experiences directly 
from first-hand sources as they distrust institutions 


e Transparency-Individuals glean available informa- 
tion (e.g. comments) to judge the incentives and 
inclinations of various Sources 


e Certainty about value of time spent-Individuals 
avoid spending time on content that does not fulfill 
their needs 


s ahay we ite, KA i 


What ney darit tell you about the COVID-19 
vaccine 


Gp Pepto aacuss COVIO-TY voccmnes 


JANUARY 
2022 


acaba | 


Dovid: 19 vaccines : The latest update 
th Jaeaiary 


Stimulus: 
Presenting users with a definite playlist of 


Stimulus; 
videos that indicates its helpfulness as that informs them about the sentiments 
sourced from ‘others’ and whose size'is contained in the comments 
defined too 


Presenting users with an information prompt 


When levers were harnessed to develop new social 
media concepts that aligned with these needs, the 
quantitative results indicate a nudge towards more 
COVID vaccine positive content and away from 
pathways that lead to vaccine hesitancy and misinfor- 
mation. There is a preference for wisdom of the tribe 
over wisdom of the crowd, as content suggestions 
sourced from people similar to themselves than 

‘other YouTube users. ‘Likes’ and ‘Views’ are signals 
not credible enough for people to evaluate content. 
Providing prompts that make audience aware of 
potential YouTube algorithmic bias is helpful in promot- 
ing vaccine positive content. 


SOLUTION 
CONCEPTS 


Screen captures 
from YouTube 
demonstrating the 
various stimulus. 


COVID-T9 QUA | Do wa aed the 
booster 19 shoti” How do we decide? 


ed 


Stimulus: 

Presenting users while they view a 
covid-19 vaccine related content, a 
suggestive:content they might miss out 
on because of YouTube's algorithm 
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The process deployed on this project encompassed 
three stages — qualitative interviews, human centered 
design solution development, and quantitative solution/ 
lever testing. Combining these techniques into one study 
provided a much richer and holistic view of the challenge 
of vaccine hesitancy. 


1: Exploration of motivations and needs 


A total of six 90-minute triad interviews (3 respondents 
together) were conducted with participants screened for the 
vaccine hesitant characteristics aligned to our project. 


Triad interviews probed motivations for the search or 
demand for alternative narratives about COVID vaccines. 
Leveraging motivation as an anchor, interviews then 
probed current YouTube Search and Consumption 
routines to expose the needs those routines address. The 
behavioral drivers revealed the core unmet needs of this 
user group in their attempts to identify a “path to safety” 
for themselves. 


EFFECTIVE NUDGES 


2: HCD-led generative solutioning 


Leveraging the dominant pattern of opinion forming 
needs and online search & judgment heuristics, we 
developed 2-3 core hypotheses for changing search and 
content judgement decisions. The top 3 were further 
developed into user scenarios/simulations brought to life 
via screen flow wireframes and/or story boards that were 
leveraged in the subsequent solution testing. 


3: Concept testing 


Outputs from phase 2 were evaluated for efficacy by sur- 
veying 500 respondents screened using the same criteria 
outlined in phase 1. Respondents were exposed to each 
concept and asked a series of close-ended questions 
about each individually and comparatively across all 3. 
The results revealed which concepts tied to the unmet 
needs of this population resulted in a stated shift/nudge 
in search or judgement behavior towards a more vaccine 
positive landscape of content. 


A graphic looking at how vaccine positive pathways could be more successful at meeting the demand for information. 


VACCINATION ACCEPTANCE 
@) RESEARCH NETWORK = = 
B came vaccnee msnu 
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APPENDICES 


APPENDIX 1: PROJECT DETAILS 


GRANTEE 


19 To Zero in partnership with the 
University of Calgary and IV.Al 


Busara Center for Behavioral 
Economics 


Center for Media Engagement at 
the University of Texas at Austin 


Corowa-kun 


Grameen Foundation India Pvt 
Limited 


Institute for Global Health 
Sciences, University of California 
San Francisco 


Institute of Tropical Medicine 
Antwerp 


International Longevity Centre 


IRD Global 


Johns Hopkins University 


Jonathan Yong Lee (Individual) - 
Stanford University 


Laboratory of Data Discovery for 
Health (D24H) 


Macro-Eyes, Inc. 


Mali Health 


Minority Rights Group 


Morning Star Lodge 


PROJECT NAME 


Empathetic Engagement with the Vaccine Hesitant in Online Spaces 


Natural language processing to systematically analyze social media sentiment among 
vulnerable populations 


Crowdsourcing content to combat misinformation in the Global South 


Leveraging moral values and social media to boost vaccine confidence and overcome 
medical mistrust and historical racism among Black populations in the U.S. and Cameroon 


Impact of a LINE chatbot and a webinar on COVID-19 vaccine intention and vaccine 
confidence in Japan: a Randomized Controlled Trial 


D-VACSI: Driving Vaccination Confidence through Social Media Innovations 


Who to trust: combating COVID-19 vaccine misinformation through trusted messengers and 
social networks in indigenous communities in Guatemala 


Cultivating Online Safe Spaces: Addressing unspoken hesitancy to build vaccine confidence 
in healthcare workers in Belgium 


Generation Vax — Leveraging intergenerational relations to increase vaccination 
uptake 


Boost: Building vibrant online communities to support COVID-19 vaccine uptake through 
participatory engagement principles 


A Social Media Approach to Young People in India Serving as COVID-19 Vaccine Advocates 


Evaluating the Frequency and Effectiveness of Common Rhetorical Strategies Used to Attack 
and Defend the Trustworthiness of Public Health Institutions 


Development and impact assessment of a conversational Al service (chatbot) on Covid-19 
vaccine confidence and uptake. 


Predicting vaccine hesitancy among eligible communities in Western Cape Province, South 
Africa 


Use of voice-based social media messages to improve vaccination knowledge and 
confidence in underserved peri-urban communities 


Diversity: Impact on Vaccine Equality (DIVE) 


Where Past Meets Present: Understanding COVID-19 Vaccine Hesitancy among Indigenous 
Peoples in Saskatchewan 


Country 
(research) 


Cc 


$ 


Canada, USA 


Kenya, 
Philippines, Nepal 


US, Cameroon 


Japan 


India 


Guatemala 


Belgium 


UK 


Pakistan 


India 


Hong Kong, 
Singapore, 
Thailand 


South Africa 


Mali 


Pakistan, Sri 
Lanka, Kenya, 
Algeria 


Canada 


Closed loop 
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Twitter 


TikTok 


WhatsApp/ 


msg 


ChatBots* 


Influencers 


Crowd- 


sourcing 


Other/ 
multiple 


GRANTEE 


Oklahoma State University 


ORB International 


Purpose Campaigns LLC 


RMIT University 


Shujaaz, Inc. 


The Behavioural Insights Team 


The Indiana University Observatory 
on Social Media 


Unity Consortium 


University of Antwerp 
University of California, San 
Francisco 


University of Chicago 


University of Chicago, Department 
of Economics 


University of North Carolina at 
Chapel Hill 


University of Washington 
(Department of Human Centered 
Design & Engineering) 


University of Washington (Institute 
for Health Metrics and Evaluation) 


Upstream Thinking 


PROJECT NAME 


Modeling Vaccine Confidence Interventions for Marginalized Migrant Communities: A Mixed 
Method Approach to Leveraging Social Media Narratives 


Exploring drivers of HPV vaccine confidence and community-led solutions leveraging the 
experiences of mothers, daughters, and health-care providers in Brazil 


Harnessing virality for vaccine confidence: testing tactics to optimize sharing of pro vaccine 
content between WhatsApp and traditional social media platforms 


Modeling vaccine-related information to create hybrid information hubs for people with 
disabilities in Vietnam and Indonesia 


Tunajengana: Using social media to mobilise young East Africans to champion community 
uptake of COVID-19 vaccines 


Designing and testing a WhatsApp chatbot to increase Covid-19 vaccination bookings in 
Argentina 


"The Black and Thriving Project": The use of message framing and social contagion to 
promote vaccines in African American communities 


Study on Activating Vaccine Supportive Parents of Adolescents and Young Adults as Trusted 
Voices on Social Media 


Research to increase confidence in maternal vaccination by digital communication 
interventions 


WhatsApp Bots for Overcoming Vaccine Misinformation and Increasing Vaccine Confidence 
among Pregnant Women in Northern India: Development and Impact Assessment 


Testing interventions to address vaccine hesitancy on Facebook in East and West Africa 


Positive reinforcement of online and offline social signaling to accelerate COVID vaccine 
uptake 


Targeted Vaccine PSAs Based on Political Science and Economic Insights 


Large-scale Studies to Advance Public’s Access and Trust of COVID-19 Vaccine Research on 
Social Media 


Using social media to identify drivers of COVID-19 vaccine hesitancy in the US by county and 
ZIP code 


Redirecting existing demand for information towards vaccine confident interpretation by 
identifying online behavioral nudge interventions 


Country 
(research) 


US 


Brazil 


Nigeria, India, 
Brazil 


Vietnam, 
Indonesia 


Kenya, Tanzania 


Argentina 


US 


US 


Belgium + 
10 European 
countries 


India 


Nigeria, Kenya 


Sierra Leone 


US 


US 


US 


US 


TOTALS 


Closed loop 
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APPENDIX 2: 
ADVISORY COUNCIL 


We want to especially thank the members of the Advisory Council — a small interdisciplinary group of 
domain experts from the vaccination confidence, public health, social media, and behavioral and data 
sciences fields — for providing expert guidance and thought leadership during key milestones of this 
Fund's grant selection and management processes. 


= 


Benjamin Djoudabaye, 
Africa CDC 


Monta Reinfelde, WHO 


Renos Vakis, World Bank Angus Thomson, UNICEF 


% SEF 


~ 
= 


Monica Jain, 3ie Impact Gustav Praekelt, Praekelt Amin Rahimian, University 
Group of Pittsburg 


AW Zz 
Sohail Agha, Global 


Chizoba Barbara Wonodi, The 
Johns Hopkins University Confidence Project South Asia Health Leader | Behavioral 


Heidi Larson, The Vaccine Shobhini Mukerji, J-PAL 


Scientist | Intervention 
Designer & Evaluator 


iJ f 
Ana Maria Munoz Boudet, 
World Bank 
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APPENDIX 3: THE FUND'S PRINCIPLES 


Priority was given to proposals that demonstrated how the Fund's core principles, as articulated below, 


would be incorporated into research plans. 


oa athp— Equity: Given the disproportionate 

A A impact COVID-19 has had on historically 
v % marginalized or excluded communities 
globally, the Fund is committed to 
ensuring that equity is core to the work we support, 
and we prioritize equity as an essential principle. 

The WHO defines equity as the absence of avoidable 
or remediable differences among groups of people, 
whether those groups are defined socially, econom- 
ically, demographically, or geographically. Equity 
includes the shifting of power to the communities of 
focus, shifting narratives and language to be represen- 
tative and inclusive of all experiences and identities. 
Thus, this Fund is interested in proposals that focus 
on reaching historically marginalized or excluded 
populations globally and are authentically considerate, 
aware of, and proactively addressing the significant 
barriers those populations experience. 


@ Interdisciplinary Collaboration: This 
if ` Fund highly encourages interdisciplinary 
© © collaboration and cross-fertilization of ideas 
“=~ from outside traditional disciplines. While 
not exhaustive, example researchers and disciplines 
that this Fund would be interested in seeing further 
integrated and supported include behavioral sciences, 
communications (including marketing), data sciences, 
technology, and public health. This Fund firmly believes 
that rigorous collaboration and partnership are at the 
heart of the truly bold and transformative approaches 
(and solutions) needed to address the complexity of 
today’s vaccination confidence and uptake challenges. 


20 \ External Validity: Challenges and solutions 
os to increasing COVID-19 vaccination con- 
O %0 fidence and uptake can often be context 
x OY’ specific. While this Fund seeks to provide 
targeted support to both underserved and 
historically excluded or marginalized communities, this 
Fund's priority is also to surface insights that are more 
universally valid and potentially scalable, at least within 
focus communities globally. In lay terms: the Fund 
would prefer to support research where the findings 
are potentially applicable to segments of the popula- 
tion or population cohorts rather than, for example, a 
single demographic within a single city. 


Community Engagement: We believe 
one way to achieve equity is ensuring 
ee authentic community engagement 

throughout this work. From research 
design through to research implementation and 
dissemination of findings, we highly encourage all 
applicants to proactively engage and authentically 
partner with the communities with whom they are 
working. This engagement would include seeking 
to understand the obstacles, potential solutions 
and assets to build on, from the perspective of the 
community — ensuring that their voices, perspectives, 
and experiences are centered in the work. This also 
includes sharing resources with the community, as 
with any other partners. The Fund will not finance 
proposals that appear to be extractive in nature vis 
a vis the communities they seek to work in or learn 
from; we very much encourage applications from 
individuals and institutions representative of these 
same communities. 


Actionable Focus: Research should be 
immediately actionable and valuable for 
the global health community rather than 
highly theoretical. However, the Fund will 
consider projects that may not directly 
increase vaccinations in a particular community, 
but rather, for example, provide new approaches 

to increasing vaccination confidence that can be 
adapted to other settings/contexts or provide new 
insights on vaccination behavior that can be shared 
with and used by others. 


Ce) 


ọ Transparency and Information Sharing: 

@ Í ® This Fund recognizes that we can only 
©- "® truly maximize our impact by being 

€ é ® transparent. Information sharing is a 

critical process; it will enable the global 

health community to collaborate effectively and 
efficiently to solve the toughest challenges around 
increasing vaccination confidence and uptake. This 
Fund is interested in supporting proposals that are 
committed to rapidly disseminating insights, data, 
and learnings from their work as public goods for 
implementers to utilize in the design and execution 
of their vaccination campaigns and to continue 
to advance the global agenda around vaccination 
confidence and uptake. Proposals that explicitly 
embrace open access and open data principles will 
be viewed favorably by the Fund. 
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ADVANCING HEALTH ONLINE 


SK For more information contact: 
tS vcfinfo@charity.org 


